Jul 20 18:13:23.033349 ip-10-0-129-84 systemd[1]: Starting Kubernetes Kubelet...
Jul 20 18:13:23.471332 ip-10-0-129-84 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 18:13:23.471332 ip-10-0-129-84 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 20 18:13:23.471332 ip-10-0-129-84 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 18:13:23.471332 ip-10-0-129-84 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 20 18:13:23.471332 ip-10-0-129-84 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 18:13:23.472029 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.471391    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 20 18:13:23.479928 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479906    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 18:13:23.479928 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479922    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 18:13:23.479928 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479927    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 18:13:23.479928 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479931    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479935    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479939    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479942    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479945    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479948    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479951    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479954    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479958    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479961    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479965    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479970    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479974    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479978    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479982    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479985    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479988    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479991    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479995    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.479998    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 18:13:23.480048 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480001    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480004    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480006    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480009    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480012    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480015    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480017    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480020    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480023    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480026    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480029    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480032    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480035    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480037    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480041    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480044    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480047    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480050    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480054    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480057    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 18:13:23.480582 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480060    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480063    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480066    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480072    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480076    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480079    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480083    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480087    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480090    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480094    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480097    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480100    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480103    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480106    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480108    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480111    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480114    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480117    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 18:13:23.481111 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480119    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480122    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480125    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480140    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480143    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480146    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480149    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480152    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480156    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480159    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480162    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480165    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480168    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480171    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480174    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480178    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480180    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480183    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480186    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480189    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 18:13:23.481576 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480191    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480194    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480197    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480200    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480203    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480205    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480208    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480211    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480213    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480216    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480219    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480221    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480224    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480227    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480230    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480233    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480235    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.480238    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481025    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481032    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 18:13:23.482072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481036    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481039    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481042    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481045    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481048    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481051    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481053    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481056    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481059    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481062    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481067    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481071    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481074    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481077    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481080    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481083    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481086    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481089    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481091    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 18:13:23.482629 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481095    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481097    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481100    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481103    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481105    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481109    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481111    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481114    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481117    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481120    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481123    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481126    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481146    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481150    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481153    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481156    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481159    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481162    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481166    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481170    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 18:13:23.483189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481173    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481177    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481180    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481183    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481186    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481189    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481191    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481194    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481197    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481200    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481203    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481207    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481211    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481214    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481218    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481221    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481224    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481227    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481229    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 18:13:23.483701 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481232    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481235    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481238    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481240    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481243    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481246    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481251    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481254    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481257    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481260    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481263    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481265    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481268    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481271    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481274    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481277    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481279    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481282    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481285    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481287    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 18:13:23.484194 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481290    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481293    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481298    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481300    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481303    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481306    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481309    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481312    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481314    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481317    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481320    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481323    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481326    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481328    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481331    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481334    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481337    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481340    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481343    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481740    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 18:13:23.484716 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481745    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481749    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481751    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481754    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481757    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481760    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481763    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481767    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481770    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481773    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481776    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481779    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481782    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481785    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481787    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481792    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481796    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481799    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481802    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 18:13:23.485229 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481806    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481809    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481812    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481815    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481818    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481820    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481824    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481827    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481829    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481832    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481834    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481838    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481841    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481845    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481847    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481850    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481853    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481855    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481858    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481861    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 18:13:23.485715 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481863    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481866    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481869    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481872    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481874    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481877    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481879    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481882    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481885    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481888    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481890    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481893    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481895    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481898    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481901    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481904    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481907    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481909    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481914    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 18:13:23.486271 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481917    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481920    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481923    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481926    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481928    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481931    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481934    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481937    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481939    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481942    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481945    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481949    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481952    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481955    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481958    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481961    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481964    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481966    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481969    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481972    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 18:13:23.486766 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481974    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481977    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481980    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481983    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481986    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481988    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481991    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481994    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.481997    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482000    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482002    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482006    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482008    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482011    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482014    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482017    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482019    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482022    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482024    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482027    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 18:13:23.487305 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482102    2569 flags.go:64] FLAG: --address="0.0.0.0"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482111    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482118    2569 flags.go:64] FLAG: --anonymous-auth="true"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482123    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482140    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482144    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482148    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482153    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482156    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482160    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482163    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482167    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482170    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482174    2569 flags.go:64] FLAG: --cgroup-root=""
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482177    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482180    2569 flags.go:64] FLAG: --client-ca-file=""
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482183    2569 flags.go:64] FLAG: --cloud-provider="external"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482187    2569 flags.go:64] FLAG: --cluster-dns="[]"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482191    2569 flags.go:64] FLAG: --cluster-domain=""
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482195    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482198    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482202    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482205    2569 flags.go:64] FLAG: --container-log-max-files="5"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482209    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 20 18:13:23.487852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482212    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482215    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482219    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482222    2569 flags.go:64] FLAG: --contention-profiling="false"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482225    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482229    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482232    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482235    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482240    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482244    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482247    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482250    2569 flags.go:64] FLAG: --enable-load-reader="false"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482253    2569 flags.go:64] FLAG: --enable-server="true"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482256    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482261    2569 flags.go:64] FLAG: --event-burst="100"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482264    2569 flags.go:64] FLAG: --event-qps="50"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482267    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482270    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482274    2569 flags.go:64] FLAG: --eviction-hard=""
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482278    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482281    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482283    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482289    2569 flags.go:64] FLAG: --eviction-soft=""
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482292    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482295    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 20 18:13:23.488485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482298    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482302    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482305    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482309    2569 flags.go:64] FLAG: --fail-swap-on="true"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482312    2569 flags.go:64] FLAG: --feature-gates=""
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482316    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482319    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482322    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482325    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482328    2569 flags.go:64] FLAG: --healthz-port="10248"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482332    2569 flags.go:64] FLAG: --help="false"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482335    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-129-84.ec2.internal"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482339    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482342    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482345    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482349    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482353    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482356    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482359    2569 flags.go:64] FLAG: --image-service-endpoint=""
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482362    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482365    2569 flags.go:64] FLAG: --kube-api-burst="100"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482368    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482372    2569 flags.go:64] FLAG: --kube-api-qps="50"
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482375    2569 flags.go:64] FLAG: --kube-reserved=""
Jul 20 18:13:23.489183 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482378    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482381    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482384    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482387    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482390    2569 flags.go:64] FLAG: --lock-file=""
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482394    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482398    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482401    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482411    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482414    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482418    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482421    2569 flags.go:64] FLAG: --logging-format="text"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482424    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482427    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482430    2569 flags.go:64] FLAG: --manifest-url=""
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482433    2569 flags.go:64] FLAG: --manifest-url-header=""
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482438    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482441    2569 flags.go:64] FLAG: --max-open-files="1000000"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482445    2569 flags.go:64] FLAG: --max-pods="110"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482448    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482451    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482455    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482458    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482462    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482465    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 20 18:13:23.489804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482469    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482476    2569 flags.go:64] FLAG: --node-status-max-images="50"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482481    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482485    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482488    2569 flags.go:64] FLAG: --pod-cidr=""
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482491    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482497    2569 flags.go:64] FLAG: --pod-manifest-path=""
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482500    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482504    2569 flags.go:64] FLAG: --pods-per-core="0"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482507    2569 flags.go:64] FLAG: --port="10250"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482510    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482513    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e6e6be6744d5d377"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482517    2569 flags.go:64] FLAG: --qos-reserved=""
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482522    2569 flags.go:64] FLAG: --read-only-port="10255"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482525    2569 flags.go:64] FLAG: --register-node="true"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482528    2569 flags.go:64] FLAG: --register-with-taints=""
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482531    2569 flags.go:64] FLAG: --registry-burst="10"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482535    2569 flags.go:64] FLAG: --registry-qps="5"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482538    2569 flags.go:64] FLAG: --reserved-cpus=""
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482541    2569 flags.go:64] FLAG: --reserved-memory=""
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482545    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482548    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482552    2569 flags.go:64] FLAG: --rotate-certificates="false"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482555    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482558    2569 flags.go:64] FLAG: --runonce="false"
Jul 20 18:13:23.490468 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482561    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482564    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482567    2569 flags.go:64] FLAG: --seccomp-default="false"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482570    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482573    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482576    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482579    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482583    2569 flags.go:64] FLAG: --storage-driver-password="root"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482586    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482591    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482594    2569 flags.go:64] FLAG: --storage-driver-user="root"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482596    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482600    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482603    2569 flags.go:64] FLAG: --system-cgroups=""
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482606    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482611    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482614    2569 flags.go:64] FLAG: --tls-cert-file=""
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482617    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482621    2569 flags.go:64] FLAG: --tls-min-version=""
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482624    2569 flags.go:64] FLAG: --tls-private-key-file=""
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482630    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482634    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482637    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482640    2569 flags.go:64] FLAG: --v="2"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482645    2569 flags.go:64] FLAG: --version="false"
Jul 20 18:13:23.491442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482650    2569 flags.go:64] FLAG: --vmodule=""
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482653    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.482657    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482777    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482781    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482785    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482788    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482791    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482794    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482798    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482803    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482806    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482809    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482812    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482815    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482818    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482823    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482826    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482828    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 18:13:23.492540 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482831    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482834    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482837    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482840    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482843    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482846    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482849    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482852    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482856    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482858    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482861    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482864    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482867    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482870    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482873    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482875    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482878    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482881    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482884    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482886    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 18:13:23.493424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482889    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482892    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482895    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482898    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482900    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482903    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482906    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482921    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482925    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482930    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482933    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482936    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482940    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482944    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482948    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482951    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482955    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482958    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482961    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 18:13:23.494426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482964    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482968    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482971    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482974    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482978    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482983    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482986    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482989    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482992    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482995    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.482997    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483000    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483003    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483006    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483009    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483011    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483014    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483017    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483020    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 18:13:23.495065 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483023    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483025    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483028    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483032    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483035    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483039    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483042    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483045    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483047    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483050    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483053    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483056    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483058    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483061    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483066    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483069    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483072    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483074    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483077    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483080    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 18:13:23.496063 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483082    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 18:13:23.496935 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483085    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 18:13:23.496935 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483088    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 18:13:23.496935 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483091    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 18:13:23.496935 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.483093    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 18:13:23.496935 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.483782    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 18:13:23.496935 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.492943    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 20 18:13:23.496935 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.492965    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 20 18:13:23.496935 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.492996    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 20 18:13:23.496935 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.493006    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 20 18:13:23.496935 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493109    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 18:13:23.496935 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493117    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 18:13:23.496935 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493123    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 18:13:23.496935 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493128    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 18:13:23.496935 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493147    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 18:13:23.496935 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493154    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493159    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493163    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493169    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493174    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493178    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493183    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493188    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493193    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493198    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493203    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493207    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493212    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493217    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493221    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493226    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493231    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493235    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493240    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493244    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493249    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 18:13:23.497369 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493253    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493258    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493263    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493269    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493275    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493280    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493285    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493289    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493294    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493299    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493303    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493308    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493312    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493317    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493322    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493327    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493332    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493336    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493341    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 18:13:23.498067 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493346    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493351    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493355    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493360    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493365    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493369    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493374    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493378    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493385    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493391    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493396    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493400    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493405    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493409    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493414    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493418    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493424    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493429    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 18:13:23.498768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493436    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493443    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493449    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493454    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493460    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493465    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493470    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493475    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493479    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493484    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493488    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493493    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493497    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493502    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493507    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493511    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493525    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493530    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493535    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493540    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 18:13:23.499259 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493544    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 18:13:23.499852 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493549    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 18:13:23.499852 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493553    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 18:13:23.499852 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493559    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 18:13:23.499852 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493563    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 18:13:23.499852 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493568    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 18:13:23.499852 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493573    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 18:13:23.499852 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493577    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 18:13:23.499852 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493582    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 18:13:23.499852 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493587    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 18:13:23.499852 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493592    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 18:13:23.499852 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493596    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 18:13:23.499852 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493601    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 18:13:23.499852 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493606    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 18:13:23.499852 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493610    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 18:13:23.499852 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493615    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 18:13:23.500322 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.493623    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 18:13:23.500322 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493832    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 18:13:23.500322 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493842    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 18:13:23.500322 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493848    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 18:13:23.500322 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493852    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 18:13:23.500322 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493857    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 18:13:23.500322 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493862    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 18:13:23.500322 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493867    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 18:13:23.500322 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493872    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 18:13:23.500322 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493877    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 18:13:23.500322 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493881    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 18:13:23.500322 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493885    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 18:13:23.500322 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493890    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 18:13:23.500322 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493896    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 18:13:23.500322 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493900    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493905    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493910    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493914    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493919    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493924    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493929    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493933    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493938    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493943    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493948    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493952    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493957    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493962    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493967    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493972    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493977    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493981    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493985    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493990    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 18:13:23.500749 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493994    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.493999    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494003    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494008    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494012    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494017    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494021    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494026    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494030    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494035    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494039    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494044    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494078    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494085    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494092    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494098    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494103    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494108    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494112    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494117    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 18:13:23.501343 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494122    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494146    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494151    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494157    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494163    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494169    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494174    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494179    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494184    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494188    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494193    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494197    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494202    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494206    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494210    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494214    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494218    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494223    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494227    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 18:13:23.501872 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494231    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494238    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494245    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494250    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494256    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494263    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494271    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494276    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494280    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494285    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494290    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494294    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494299    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494303    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494308    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494313    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494317    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494322    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494327    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 18:13:23.502705 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494331    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 18:13:23.503534 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494335    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 18:13:23.503534 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494340    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 18:13:23.503534 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494344    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 18:13:23.503534 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494349    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 18:13:23.503534 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494354    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 18:13:23.503534 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494358    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 18:13:23.503534 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:23.494363    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 18:13:23.503534 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.494372    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 18:13:23.503534 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.494557    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 20 18:13:23.503534 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.497607    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 20 18:13:23.503534 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.499359    2569 server.go:1019] "Starting client certificate rotation"
Jul 20 18:13:23.503534 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.499474    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 18:13:23.503534 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.500231    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 18:13:23.503534 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.502694    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 18:13:23.508098 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.508076    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 18:13:23.520581 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.520563    2569 log.go:25] "Validated CRI v1 runtime API"
Jul 20 18:13:23.522810 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.522763    2569 log.go:25] "Validated CRI v1 image API"
Jul 20 18:13:23.523627 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.523611    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 20 18:13:23.527400 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.527376    2569 fs.go:135] Filesystem UUIDs: map[2e581796-09be-42fd-bd04-413f504c9f74:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 91274d6a-1e6a-459c-8396-10e26ed02867:/dev/nvme0n1p4]
Jul 20 18:13:23.527461 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.527403    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 20 18:13:23.531957 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.531934    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 18:13:23.534289 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.534169    2569 manager.go:216] Machine: {Timestamp:2026-07-20 18:13:23.532581443 +0000 UTC m=+0.366553846 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095769 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25e24d3d41f17d0f130edbf3dc61aa SystemUUID:ec25e24d-3d41-f17d-0f13-0edbf3dc61aa BootID:cdc2e053-befd-4242-b8a9-349faac6b4ba Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2b:2c:37:b9:17 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2b:2c:37:b9:17 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:69:a6:07:e4:e3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 20 18:13:23.534289 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.534288    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 20 18:13:23.534402 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.534375    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 20 18:13:23.539110 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.538886    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 20 18:13:23.539284 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.539115    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-84.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 20 18:13:23.539987 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.539976    2569 topology_manager.go:138] "Creating topology manager with none policy"
Jul 20 18:13:23.540019 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.539991    2569 container_manager_linux.go:309] "Creating device plugin manager"
Jul 20 18:13:23.540019 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.539998    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 18:13:23.540752 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.540740    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 18:13:23.542096 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.542085    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 20 18:13:23.545697 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.545685    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 18:13:23.545837 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.545826    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 20 18:13:23.546453 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.546440    2569 kubelet.go:486] "Attempting to sync node with API server"
Jul 20 18:13:23.546530 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.546460    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 20 18:13:23.546530 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.546480    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 20 18:13:23.546530 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.546492    2569 kubelet.go:398] "Adding apiserver pod source"
Jul 20 18:13:23.546530 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.546502    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 20 18:13:23.546898 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.546886    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 18:13:23.546933 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.546905    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 18:13:23.550286 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.550264    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 20 18:13:23.552745 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.552727    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 20 18:13:23.552834 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.552769    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 20 18:13:23.553993 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.553978    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 20 18:13:23.554081 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.554001    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 20 18:13:23.554081 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.554012    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 20 18:13:23.554081 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.554021    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 20 18:13:23.554081 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.554036    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 20 18:13:23.554081 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.554046    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 20 18:13:23.554081 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.554059    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 20 18:13:23.554081 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.554068    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 20 18:13:23.554081 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.554081    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 20 18:13:23.554357 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.554091    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 20 18:13:23.554357 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.554105    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 20 18:13:23.554357 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.554114    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 20 18:13:23.555164 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.555152    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 20 18:13:23.555222 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.555167    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 20 18:13:23.555934 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.555911    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pq7rp"
Jul 20 18:13:23.556392 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:23.556364    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-84.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 20 18:13:23.556483 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:23.556371    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 20 18:13:23.558557 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.558537    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-84.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 20 18:13:23.558809 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.558789    2569 server.go:1268] "Started kubelet"
Jul 20 18:13:23.558904 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.558850    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 20 18:13:23.559022 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.558928    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 20 18:13:23.559110 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.559041    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 20 18:13:23.559893 ip-10-0-129-84 systemd[1]: Started Kubernetes Kubelet.
Jul 20 18:13:23.560272 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.560255    2569 server.go:310] "Adding debug handlers to kubelet server"
Jul 20 18:13:23.561077 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.560983    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 20 18:13:23.564161 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.564143    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pq7rp"
Jul 20 18:13:23.565474 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:23.564435    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-84.ec2.internal.18c4120df8829805  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-84.ec2.internal,UID:ip-10-0-129-84.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-84.ec2.internal,},FirstTimestamp:2026-07-20 18:13:23.558742021 +0000 UTC m=+0.392714403,LastTimestamp:2026-07-20 18:13:23.558742021 +0000 UTC m=+0.392714403,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-84.ec2.internal,}"
Jul 20 18:13:23.566011 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:23.565994    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 20 18:13:23.568242 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.568228    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 20 18:13:23.568288 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.568245    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 20 18:13:23.568480 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:23.568427    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-84.ec2.internal\" not found"
Jul 20 18:13:23.568898 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.568880    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 20 18:13:23.568898 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.568900    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 20 18:13:23.569113 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.568884    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 20 18:13:23.569113 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.569009    2569 reconstruct.go:97] "Volume reconstruction finished"
Jul 20 18:13:23.569113 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.569018    2569 reconciler.go:29] "Reconciler: start to sync state"
Jul 20 18:13:23.569382 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.569361    2569 factory.go:55] Registering systemd factory
Jul 20 18:13:23.569382 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.569384    2569 factory.go:223] Registration of the systemd container factory successfully
Jul 20 18:13:23.569655 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.569640    2569 factory.go:153] Registering CRI-O factory
Jul 20 18:13:23.569735 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.569659    2569 factory.go:223] Registration of the crio container factory successfully
Jul 20 18:13:23.569735 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.569719    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 20 18:13:23.569844 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.569745    2569 factory.go:103] Registering Raw factory
Jul 20 18:13:23.569844 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.569763    2569 manager.go:1196] Started watching for new ooms in manager
Jul 20 18:13:23.570292 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.570266    2569 manager.go:349] Starting recovery of all containers
Jul 20 18:13:23.574913 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.574884    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 18:13:23.579327 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:23.579303    2569 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-84.ec2.internal\" not found" node="ip-10-0-129-84.ec2.internal"
Jul 20 18:13:23.583155 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.583118    2569 manager.go:354] Recovery completed
Jul 20 18:13:23.588375 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.588362    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 18:13:23.590296 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.590277    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 18:13:23.590378 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.590313    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 18:13:23.590378 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.590328    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasSufficientPID"
Jul 20 18:13:23.591341 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.591326    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 20 18:13:23.591341 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.591339    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 20 18:13:23.591465 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.591349    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 18:13:23.591465 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.591356    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 18:13:23.592081 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.592051    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 18:13:23.592181 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.592091    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 18:13:23.592181 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.592101    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasSufficientPID"
Jul 20 18:13:23.592628 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.592615    2569 policy_none.go:49] "None policy: Start"
Jul 20 18:13:23.592666 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.592633    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 20 18:13:23.592666 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.592644    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 20 18:13:23.593915 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.593904    2569 policy_none.go:47] "Start"
Jul 20 18:13:23.625743 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.625589    2569 manager.go:337] "Starting Device Plugin manager"
Jul 20 18:13:23.625847 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:23.625781    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 20 18:13:23.625847 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.625795    2569 server.go:87] "Starting device plugin registration server"
Jul 20 18:13:23.626068 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.626056    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 20 18:13:23.626106 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.626072    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 20 18:13:23.626219 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.626193    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 20 18:13:23.626317 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.626287    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 20 18:13:23.626317 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.626298    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 20 18:13:23.626980 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:23.626962    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 20 18:13:23.627072 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:23.627008    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-84.ec2.internal\" not found"
Jul 20 18:13:23.727296 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.727258    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 18:13:23.728684 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.728424    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 18:13:23.728684 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.728458    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 18:13:23.728684 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.728473    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasSufficientPID"
Jul 20 18:13:23.728684 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.728508    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-84.ec2.internal"
Jul 20 18:13:23.737564 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.737549    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-84.ec2.internal"
Jul 20 18:13:23.737625 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:23.737572    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-84.ec2.internal\": node \"ip-10-0-129-84.ec2.internal\" not found"
Jul 20 18:13:23.750960 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.750916    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 20 18:13:23.752292 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.752262    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 20 18:13:23.752410 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.752302    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 20 18:13:23.752410 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.752324    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 20 18:13:23.752410 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:23.752359    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 20 18:13:23.754641 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.754616    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 18:13:23.758817 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:23.758794    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-84.ec2.internal\" not found"
Jul 20 18:13:23.855720 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.855635    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-84.ec2.internal"]
Jul 20 18:13:23.855842 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.855726    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 18:13:23.856842 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.856826    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 18:13:23.856908 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.856859    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 18:13:23.856908 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.856876    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasSufficientPID"
Jul 20 18:13:23.858379 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.858364    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 18:13:23.858540 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.858526    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal"
Jul 20 18:13:23.858585 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.858561    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 18:13:23.858860 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:23.858842    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-84.ec2.internal\" not found"
Jul 20 18:13:23.859176 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.859158    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 18:13:23.859329 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.859193    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 18:13:23.859329 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.859159    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 18:13:23.859329 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.859236    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 18:13:23.859329 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.859249    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasSufficientPID"
Jul 20 18:13:23.859329 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.859204    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasSufficientPID"
Jul 20 18:13:23.860628 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.860611    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-84.ec2.internal"
Jul 20 18:13:23.860720 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.860644    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 18:13:23.861364 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.861349    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 18:13:23.861431 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.861370    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 18:13:23.861431 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.861384    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeHasSufficientPID"
Jul 20 18:13:23.870222 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.870203    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/873a0c6273e20da2e23695f53316d6b7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal\" (UID: \"873a0c6273e20da2e23695f53316d6b7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal"
Jul 20 18:13:23.870319 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.870234    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/873a0c6273e20da2e23695f53316d6b7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal\" (UID: \"873a0c6273e20da2e23695f53316d6b7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal"
Jul 20 18:13:23.870319 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.870261    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/409747c0910991761a210f0528443183-config\") pod \"kube-apiserver-proxy-ip-10-0-129-84.ec2.internal\" (UID: \"409747c0910991761a210f0528443183\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-84.ec2.internal"
Jul 20 18:13:23.890904 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:23.890885    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-84.ec2.internal\" not found" node="ip-10-0-129-84.ec2.internal"
Jul 20 18:13:23.895684 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:23.895667    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-84.ec2.internal\" not found" node="ip-10-0-129-84.ec2.internal"
Jul 20 18:13:23.959191 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:23.959162    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-84.ec2.internal\" not found"
Jul 20 18:13:23.970918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.970898    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/873a0c6273e20da2e23695f53316d6b7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal\" (UID: \"873a0c6273e20da2e23695f53316d6b7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal"
Jul 20 18:13:23.971018 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.970927    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/873a0c6273e20da2e23695f53316d6b7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal\" (UID: \"873a0c6273e20da2e23695f53316d6b7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal"
Jul 20 18:13:23.971018 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.970945    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/409747c0910991761a210f0528443183-config\") pod \"kube-apiserver-proxy-ip-10-0-129-84.ec2.internal\" (UID: \"409747c0910991761a210f0528443183\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-84.ec2.internal"
Jul 20 18:13:23.971018 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.970974    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/409747c0910991761a210f0528443183-config\") pod \"kube-apiserver-proxy-ip-10-0-129-84.ec2.internal\" (UID: \"409747c0910991761a210f0528443183\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-84.ec2.internal"
Jul 20 18:13:23.971018 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.970995    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/873a0c6273e20da2e23695f53316d6b7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal\" (UID: \"873a0c6273e20da2e23695f53316d6b7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal"
Jul 20 18:13:23.971018 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:23.971011    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/873a0c6273e20da2e23695f53316d6b7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal\" (UID: \"873a0c6273e20da2e23695f53316d6b7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal"
Jul 20 18:13:24.060051 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:24.060013    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-84.ec2.internal\" not found"
Jul 20 18:13:24.166677 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:24.166606    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-84.ec2.internal\" not found"
Jul 20 18:13:24.195145 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:24.195119    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal"
Jul 20 18:13:24.198284 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:24.198263    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-84.ec2.internal"
Jul 20 18:13:24.266741 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:24.266704    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-84.ec2.internal\" not found"
Jul 20 18:13:24.372974 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:24.372939    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-84.ec2.internal\" not found"
Jul 20 18:13:24.474020 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:24.473951    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-84.ec2.internal\" not found"
Jul 20 18:13:24.499127 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:24.499105    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 20 18:13:24.499287 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:24.499265    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 20 18:13:24.499327 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:24.499280    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 20 18:13:24.569630 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:24.569583    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 18:08:24 +0000 UTC" deadline="2028-03-06 22:16:50.732474352 +0000 UTC"
Jul 20 18:13:24.569630 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:24.569621    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14284h3m26.162857866s"
Jul 20 18:13:24.574876 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:24.574856    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-84.ec2.internal\" not found"
Jul 20 18:13:24.675144 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:24.675099    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-84.ec2.internal\" not found"
Jul 20 18:13:24.738622 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:24.738573    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod409747c0910991761a210f0528443183.slice/crio-e51797f213ae45d94378212cb722c05521ebd2bf5ced50f5f3e3079a3f3d9ba1 WatchSource:0}: Error finding container e51797f213ae45d94378212cb722c05521ebd2bf5ced50f5f3e3079a3f3d9ba1: Status 404 returned error can't find the container with id e51797f213ae45d94378212cb722c05521ebd2bf5ced50f5f3e3079a3f3d9ba1
Jul 20 18:13:24.743336 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:24.743314    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 18:13:24.755359 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:24.755310    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-84.ec2.internal" event={"ID":"409747c0910991761a210f0528443183","Type":"ContainerStarted","Data":"e51797f213ae45d94378212cb722c05521ebd2bf5ced50f5f3e3079a3f3d9ba1"}
Jul 20 18:13:24.762635 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:24.762612    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod873a0c6273e20da2e23695f53316d6b7.slice/crio-484a552f420edc6d6df42220b98afa88cb965b314ed6c0d33e7f6336ffb87a3b WatchSource:0}: Error finding container 484a552f420edc6d6df42220b98afa88cb965b314ed6c0d33e7f6336ffb87a3b: Status 404 returned error can't find the container with id 484a552f420edc6d6df42220b98afa88cb965b314ed6c0d33e7f6336ffb87a3b
Jul 20 18:13:24.775390 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:24.775362    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-84.ec2.internal\" not found"
Jul 20 18:13:24.807019 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:24.806989    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 18:13:24.829650 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:24.829623    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 18:13:24.869512 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:24.869486    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-84.ec2.internal"
Jul 20 18:13:24.878301 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:24.878277    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 18:13:24.879403 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:24.879389    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal"
Jul 20 18:13:24.889755 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:24.889732    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 18:13:25.547375 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.547336    2569 apiserver.go:52] "Watching apiserver"
Jul 20 18:13:25.555337 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.555097    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 20 18:13:25.557652 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.557617    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-vc9wm","openshift-multus/network-metrics-daemon-g4z7w","kube-system/global-pull-secret-syncer-pjhww","kube-system/kube-apiserver-proxy-ip-10-0-129-84.ec2.internal","openshift-cluster-node-tuning-operator/tuned-z242x","openshift-image-registry/node-ca-lkqb9","openshift-network-diagnostics/network-check-target-hb8sv","openshift-network-operator/iptables-alerter-xdtrj","openshift-ovn-kubernetes/ovnkube-node-8f6m2","kube-system/konnectivity-agent-j9b5n","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27","openshift-dns/node-resolver-8bf9p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal","openshift-multus/multus-additional-cni-plugins-pq7xn"]
Jul 20 18:13:25.559749 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.559275    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j9b5n"
Jul 20 18:13:25.562503 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.561755    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.562847 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.562778    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 20 18:13:25.563450 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.563077    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-k9zql\""
Jul 20 18:13:25.563450 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.563304    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 20 18:13:25.563621 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.563503    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:25.563621 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:25.563566    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:25.564612 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.564122    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.569161 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.565954    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 20 18:13:25.569161 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.566189    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 20 18:13:25.569161 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.566389    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jkf4p\""
Jul 20 18:13:25.569161 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.566633    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 20 18:13:25.569161 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.567028    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 20 18:13:25.569161 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.567236    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 20 18:13:25.569161 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.567518    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mhhpr\""
Jul 20 18:13:25.569161 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.567821    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 20 18:13:25.569161 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.568042    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 20 18:13:25.569161 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.568056    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 20 18:13:25.569161 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.568337    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.569161 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.568926    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lkqb9"
Jul 20 18:13:25.569854 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.569280    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 20 18:13:25.569854 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.569346    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.571379 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.570132    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xdtrj"
Jul 20 18:13:25.571785 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.571759    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:25.571882 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:25.571833    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:25.573226 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.573204    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.574938 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.574891    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:25.575007 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:25.574948    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:25.577108 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.576530    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 20 18:13:25.577108 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.576685    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8bf9p"
Jul 20 18:13:25.577108 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.576908    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 20 18:13:25.578765 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.577171    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-775jc\""
Jul 20 18:13:25.578765 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.577330    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 20 18:13:25.581034 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.581011    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2c72dbe-b803-492b-9046-eeeacf69b9cf-cnibin\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.581157 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.581050    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/80fea0b1-03fb-4254-be13-4315a5fc61cc-iptables-alerter-script\") pod \"iptables-alerter-xdtrj\" (UID: \"80fea0b1-03fb-4254-be13-4315a5fc61cc\") " pod="openshift-network-operator/iptables-alerter-xdtrj"
Jul 20 18:13:25.581157 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.581146    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-log-socket\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.581279 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.581178    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-env-overrides\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.581279 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.581208    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-registration-dir\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.581279 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.581234    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq75h\" (UniqueName: \"kubernetes.io/projected/25871790-7549-4c7b-ae62-759f32bf2c36-kube-api-access-wq75h\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.581279 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.581259    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31d6832c-6d75-41e8-b272-7edeb41be490-kubelet-config\") pod \"global-pull-secret-syncer-pjhww\" (UID: \"31d6832c-6d75-41e8-b272-7edeb41be490\") " pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:25.581492 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.581284    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2c72dbe-b803-492b-9046-eeeacf69b9cf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.581492 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.581310    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-kubernetes\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.581492 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.581358    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-run-openvswitch\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.587336 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.582934    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w285p\""
Jul 20 18:13:25.587336 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.583217    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 20 18:13:25.587336 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.583449    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 20 18:13:25.587336 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.583681    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 20 18:13:25.587336 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.584533    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 20 18:13:25.587336 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.584783    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 20 18:13:25.587336 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.585170    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 20 18:13:25.587336 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.585391    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 20 18:13:25.587336 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.585609    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 20 18:13:25.587336 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.586385    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 20 18:13:25.587336 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.586619    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-597tm\""
Jul 20 18:13:25.587336 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.586929    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 20 18:13:25.589004 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.588301    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 20 18:13:25.589004 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.588336    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cg6j5\""
Jul 20 18:13:25.589004 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.588621    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 20 18:13:25.589004 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.588628    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 20 18:13:25.589004 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.588850    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-ovn-node-metrics-cert\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.589004 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.588868    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6kvwq\""
Jul 20 18:13:25.589004 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.588901    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-device-dir\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.589004 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.588941    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-systemd\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.590541 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.588972    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-run\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.590541 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.590157    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5a1608a-991c-4c92-978f-7f00e84fda46-host\") pod \"node-ca-lkqb9\" (UID: \"d5a1608a-991c-4c92-978f-7f00e84fda46\") " pod="openshift-image-registry/node-ca-lkqb9"
Jul 20 18:13:25.590541 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.590194    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.590541 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.590232    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ec1b495c-bcb6-493b-811a-2e0957afa7a6-konnectivity-ca\") pod \"konnectivity-agent-j9b5n\" (UID: \"ec1b495c-bcb6-493b-811a-2e0957afa7a6\") " pod="kube-system/konnectivity-agent-j9b5n"
Jul 20 18:13:25.590541 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.590266    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2c72dbe-b803-492b-9046-eeeacf69b9cf-os-release\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.590541 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.589689    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5scjm\""
Jul 20 18:13:25.590541 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.590299    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j492\" (UniqueName: \"kubernetes.io/projected/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-kube-api-access-2j492\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.590541 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.590334    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/80fea0b1-03fb-4254-be13-4315a5fc61cc-host-slash\") pod \"iptables-alerter-xdtrj\" (UID: \"80fea0b1-03fb-4254-be13-4315a5fc61cc\") " pod="openshift-network-operator/iptables-alerter-xdtrj"
Jul 20 18:13:25.590541 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.590362    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-run-systemd\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.590541 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.590395    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-socket-dir\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.590541 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.589614    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 20 18:13:25.590541 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.590427    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-sys-fs\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.590541 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.590529    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f2c72dbe-b803-492b-9046-eeeacf69b9cf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.591341 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.590562    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-kubelet\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.591341 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.590593    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-sysctl-conf\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.591341 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.590622    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-sys\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.592203 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592178    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-tmp\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.592300 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592224    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-slash\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.592300 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592251    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-node-log\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.592300 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592275    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-sysconfig\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.592476 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592302    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ec1b495c-bcb6-493b-811a-2e0957afa7a6-agent-certs\") pod \"konnectivity-agent-j9b5n\" (UID: \"ec1b495c-bcb6-493b-811a-2e0957afa7a6\") " pod="kube-system/konnectivity-agent-j9b5n"
Jul 20 18:13:25.592476 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592327    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg2ld\" (UniqueName: \"kubernetes.io/projected/d5a1608a-991c-4c92-978f-7f00e84fda46-kube-api-access-jg2ld\") pod \"node-ca-lkqb9\" (UID: \"d5a1608a-991c-4c92-978f-7f00e84fda46\") " pod="openshift-image-registry/node-ca-lkqb9"
Jul 20 18:13:25.592476 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592353    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-etc-openvswitch\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.592476 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592377    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-etc-selinux\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.592476 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592402    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-lib-modules\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.592476 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592426    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-tuned\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.592476 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592452    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-cni-bin\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.592830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592475    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-host\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.592830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592502    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dc2z\" (UniqueName: \"kubernetes.io/projected/f2c72dbe-b803-492b-9046-eeeacf69b9cf-kube-api-access-6dc2z\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.592830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592528    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d5a1608a-991c-4c92-978f-7f00e84fda46-serviceca\") pod \"node-ca-lkqb9\" (UID: \"d5a1608a-991c-4c92-978f-7f00e84fda46\") " pod="openshift-image-registry/node-ca-lkqb9"
Jul 20 18:13:25.592830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592551    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-run-netns\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.592830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592577    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret\") pod \"global-pull-secret-syncer-pjhww\" (UID: \"31d6832c-6d75-41e8-b272-7edeb41be490\") " pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:25.592830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592605    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f2c72dbe-b803-492b-9046-eeeacf69b9cf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.592830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592632    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-var-lib-openvswitch\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.592830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592657    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-run-ovn\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.592830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592682    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-ovnkube-script-lib\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.592830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592708    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2c72dbe-b803-492b-9046-eeeacf69b9cf-system-cni-dir\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.592830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592735    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2c72dbe-b803-492b-9046-eeeacf69b9cf-cni-binary-copy\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.592830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592761    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-var-lib-kubelet\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.592830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592794    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-run-ovn-kubernetes\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.592830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592820    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-cni-netd\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.593590 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592843    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-ovnkube-config\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.593590 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592870    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.593590 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592902    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8nlk\" (UniqueName: \"kubernetes.io/projected/80fea0b1-03fb-4254-be13-4315a5fc61cc-kube-api-access-b8nlk\") pod \"iptables-alerter-xdtrj\" (UID: \"80fea0b1-03fb-4254-be13-4315a5fc61cc\") " pod="openshift-network-operator/iptables-alerter-xdtrj"
Jul 20 18:13:25.593590 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592944    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-modprobe-d\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.593590 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.592977    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-sysctl-d\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.593590 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.593019    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpdxr\" (UniqueName: \"kubernetes.io/projected/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-kube-api-access-zpdxr\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.593590 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.593209    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-systemd-units\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.593590 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.593264    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31d6832c-6d75-41e8-b272-7edeb41be490-dbus\") pod \"global-pull-secret-syncer-pjhww\" (UID: \"31d6832c-6d75-41e8-b272-7edeb41be490\") " pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:25.596996 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.596965    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 18:13:25.604990 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.604967    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 18:13:25.614892 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.614630    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 18:13:25.627708 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.627626    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8vqz9"
Jul 20 18:13:25.636641 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.636572    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8vqz9"
Jul 20 18:13:25.670330 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.670308    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 20 18:13:25.693550 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693514    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-node-log\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.693700 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693561    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-var-lib-cni-bin\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.693700 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693590    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-var-lib-cni-multus\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.693700 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693618    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-run-multus-certs\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.693700 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693646    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-sysconfig\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.693700 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693671    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ec1b495c-bcb6-493b-811a-2e0957afa7a6-agent-certs\") pod \"konnectivity-agent-j9b5n\" (UID: \"ec1b495c-bcb6-493b-811a-2e0957afa7a6\") " pod="kube-system/konnectivity-agent-j9b5n"
Jul 20 18:13:25.693700 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693697    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg2ld\" (UniqueName: \"kubernetes.io/projected/d5a1608a-991c-4c92-978f-7f00e84fda46-kube-api-access-jg2ld\") pod \"node-ca-lkqb9\" (UID: \"d5a1608a-991c-4c92-978f-7f00e84fda46\") " pod="openshift-image-registry/node-ca-lkqb9"
Jul 20 18:13:25.693984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693722    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-etc-openvswitch\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.693984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693747    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-multus-conf-dir\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.693984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693775    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d755879-19d5-463b-bd98-83a890d00721-multus-daemon-config\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.693984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693815    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmtrm\" (UniqueName: \"kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm\") pod \"network-check-target-hb8sv\" (UID: \"dd6d9efb-9a7b-4247-95c0-e8a3a56600eb\") " pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:25.693984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693841    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-etc-selinux\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.693984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693867    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-lib-modules\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.693984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693887    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-tuned\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.693984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693902    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-cni-bin\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.693984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693929    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-cnibin\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.693984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693955    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-hostroot\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.693984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693971    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-host\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.693984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.693986    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-run-netns\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694014    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dc2z\" (UniqueName: \"kubernetes.io/projected/f2c72dbe-b803-492b-9046-eeeacf69b9cf-kube-api-access-6dc2z\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694041    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d5a1608a-991c-4c92-978f-7f00e84fda46-serviceca\") pod \"node-ca-lkqb9\" (UID: \"d5a1608a-991c-4c92-978f-7f00e84fda46\") " pod="openshift-image-registry/node-ca-lkqb9"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694063    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-run-netns\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694083    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-os-release\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694105    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-var-lib-kubelet\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694133    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncrt4\" (UniqueName: \"kubernetes.io/projected/6d755879-19d5-463b-bd98-83a890d00721-kube-api-access-ncrt4\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694152    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret\") pod \"global-pull-secret-syncer-pjhww\" (UID: \"31d6832c-6d75-41e8-b272-7edeb41be490\") " pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694178    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f2c72dbe-b803-492b-9046-eeeacf69b9cf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694199    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-var-lib-openvswitch\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694216    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-run-ovn\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694233    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-ovnkube-script-lib\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694255    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-system-cni-dir\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694281    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2c72dbe-b803-492b-9046-eeeacf69b9cf-system-cni-dir\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694315    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2c72dbe-b803-492b-9046-eeeacf69b9cf-cni-binary-copy\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694332    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-var-lib-kubelet\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694364    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-run-ovn-kubernetes\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.694543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694391    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-cni-netd\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694417    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-ovnkube-config\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694455    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8fch\" (UniqueName: \"kubernetes.io/projected/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-kube-api-access-c8fch\") pod \"network-metrics-daemon-g4z7w\" (UID: \"1bae9e54-a7d1-46ba-9bb6-9602c8d53962\") " pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694483    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694510    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8nlk\" (UniqueName: \"kubernetes.io/projected/80fea0b1-03fb-4254-be13-4315a5fc61cc-kube-api-access-b8nlk\") pod \"iptables-alerter-xdtrj\" (UID: \"80fea0b1-03fb-4254-be13-4315a5fc61cc\") " pod="openshift-network-operator/iptables-alerter-xdtrj"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694537    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-modprobe-d\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694563    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-sysctl-d\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694591    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpdxr\" (UniqueName: \"kubernetes.io/projected/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-kube-api-access-zpdxr\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694617    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-systemd-units\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694644    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31d6832c-6d75-41e8-b272-7edeb41be490-dbus\") pod \"global-pull-secret-syncer-pjhww\" (UID: \"31d6832c-6d75-41e8-b272-7edeb41be490\") " pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694682    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/17e7c5f3-421c-4bdb-9e4e-67586149b43d-hosts-file\") pod \"node-resolver-8bf9p\" (UID: \"17e7c5f3-421c-4bdb-9e4e-67586149b43d\") " pod="openshift-dns/node-resolver-8bf9p"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694710    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2c72dbe-b803-492b-9046-eeeacf69b9cf-cnibin\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694732    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/80fea0b1-03fb-4254-be13-4315a5fc61cc-iptables-alerter-script\") pod \"iptables-alerter-xdtrj\" (UID: \"80fea0b1-03fb-4254-be13-4315a5fc61cc\") " pod="openshift-network-operator/iptables-alerter-xdtrj"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694764    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-log-socket\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694781    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-env-overrides\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694803    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-registration-dir\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694833    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq75h\" (UniqueName: \"kubernetes.io/projected/25871790-7549-4c7b-ae62-759f32bf2c36-kube-api-access-wq75h\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.695365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694860    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31d6832c-6d75-41e8-b272-7edeb41be490-kubelet-config\") pod \"global-pull-secret-syncer-pjhww\" (UID: \"31d6832c-6d75-41e8-b272-7edeb41be490\") " pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694890    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2c72dbe-b803-492b-9046-eeeacf69b9cf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694919    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-kubernetes\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694947    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-run-openvswitch\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.694973    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-ovn-node-metrics-cert\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695001    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17e7c5f3-421c-4bdb-9e4e-67586149b43d-tmp-dir\") pod \"node-resolver-8bf9p\" (UID: \"17e7c5f3-421c-4bdb-9e4e-67586149b43d\") " pod="openshift-dns/node-resolver-8bf9p"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695031    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-device-dir\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695059    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-systemd\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695083    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-run\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695109    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5a1608a-991c-4c92-978f-7f00e84fda46-host\") pod \"node-ca-lkqb9\" (UID: \"d5a1608a-991c-4c92-978f-7f00e84fda46\") " pod="openshift-image-registry/node-ca-lkqb9"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695153    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695182    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-multus-cni-dir\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695217    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-multus-socket-dir-parent\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695246    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ec1b495c-bcb6-493b-811a-2e0957afa7a6-konnectivity-ca\") pod \"konnectivity-agent-j9b5n\" (UID: \"ec1b495c-bcb6-493b-811a-2e0957afa7a6\") " pod="kube-system/konnectivity-agent-j9b5n"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695279    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2c72dbe-b803-492b-9046-eeeacf69b9cf-os-release\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695308    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j492\" (UniqueName: \"kubernetes.io/projected/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-kube-api-access-2j492\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695339    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrr9b\" (UniqueName: \"kubernetes.io/projected/17e7c5f3-421c-4bdb-9e4e-67586149b43d-kube-api-access-jrr9b\") pod \"node-resolver-8bf9p\" (UID: \"17e7c5f3-421c-4bdb-9e4e-67586149b43d\") " pod="openshift-dns/node-resolver-8bf9p"
Jul 20 18:13:25.696198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695368    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-etc-kubernetes\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695398    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/80fea0b1-03fb-4254-be13-4315a5fc61cc-host-slash\") pod \"iptables-alerter-xdtrj\" (UID: \"80fea0b1-03fb-4254-be13-4315a5fc61cc\") " pod="openshift-network-operator/iptables-alerter-xdtrj"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695431    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-run-systemd\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695461    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-socket-dir\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695493    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-sys-fs\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695525    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f2c72dbe-b803-492b-9046-eeeacf69b9cf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695552    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-kubelet\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695577    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs\") pod \"network-metrics-daemon-g4z7w\" (UID: \"1bae9e54-a7d1-46ba-9bb6-9602c8d53962\") " pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695599    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d755879-19d5-463b-bd98-83a890d00721-cni-binary-copy\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695622    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-run-k8s-cni-cncf-io\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695641    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-sysctl-conf\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695658    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-sys\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695675    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-tmp\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695692    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-slash\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695808    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-slash\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695861    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-node-log\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.695931    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-sysconfig\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.697032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.696362    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 20 18:13:25.697951 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.697354    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-etc-openvswitch\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.697951 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.697506    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-etc-selinux\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.697951 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.697622    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-lib-modules\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.697951 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.697671    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-log-socket\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.698177    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-env-overrides\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.698197    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/80fea0b1-03fb-4254-be13-4315a5fc61cc-iptables-alerter-script\") pod \"iptables-alerter-xdtrj\" (UID: \"80fea0b1-03fb-4254-be13-4315a5fc61cc\") " pod="openshift-network-operator/iptables-alerter-xdtrj"
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.698275    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-registration-dir\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.698514    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-cni-bin\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.698555    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31d6832c-6d75-41e8-b272-7edeb41be490-kubelet-config\") pod \"global-pull-secret-syncer-pjhww\" (UID: \"31d6832c-6d75-41e8-b272-7edeb41be490\") " pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:25.698641    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:25.698719    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret podName:31d6832c-6d75-41e8-b272-7edeb41be490 nodeName:}" failed. No retries permitted until 2026-07-20 18:13:26.198683049 +0000 UTC m=+3.032655421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret") pod "global-pull-secret-syncer-pjhww" (UID: "31d6832c-6d75-41e8-b272-7edeb41be490") : object "kube-system"/"original-pull-secret" not registered
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.698999    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2c72dbe-b803-492b-9046-eeeacf69b9cf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.699064    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-kubernetes\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.699100    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f2c72dbe-b803-492b-9046-eeeacf69b9cf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.699110    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-run-openvswitch\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.699196    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-var-lib-openvswitch\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.699246    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-run-ovn\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.699269    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d5a1608a-991c-4c92-978f-7f00e84fda46-serviceca\") pod \"node-ca-lkqb9\" (UID: \"d5a1608a-991c-4c92-978f-7f00e84fda46\") " pod="openshift-image-registry/node-ca-lkqb9"
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.699342    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-run-netns\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.699521    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.701918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.699578    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2c72dbe-b803-492b-9046-eeeacf69b9cf-system-cni-dir\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.699756    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-ovnkube-script-lib\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.699912    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-modprobe-d\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.700067    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31d6832c-6d75-41e8-b272-7edeb41be490-dbus\") pod \"global-pull-secret-syncer-pjhww\" (UID: \"31d6832c-6d75-41e8-b272-7edeb41be490\") " pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.700115    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-systemd-units\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.700184    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-cni-netd\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.700235    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-var-lib-kubelet\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.700281    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-run-ovn-kubernetes\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.699951    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-sysctl-d\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.700643    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2c72dbe-b803-492b-9046-eeeacf69b9cf-cni-binary-copy\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.700727    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/80fea0b1-03fb-4254-be13-4315a5fc61cc-host-slash\") pod \"iptables-alerter-xdtrj\" (UID: \"80fea0b1-03fb-4254-be13-4315a5fc61cc\") " pod="openshift-network-operator/iptables-alerter-xdtrj"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.700783    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ec1b495c-bcb6-493b-811a-2e0957afa7a6-agent-certs\") pod \"konnectivity-agent-j9b5n\" (UID: \"ec1b495c-bcb6-493b-811a-2e0957afa7a6\") " pod="kube-system/konnectivity-agent-j9b5n"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.700829    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-device-dir\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.700896    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5a1608a-991c-4c92-978f-7f00e84fda46-host\") pod \"node-ca-lkqb9\" (UID: \"d5a1608a-991c-4c92-978f-7f00e84fda46\") " pod="openshift-image-registry/node-ca-lkqb9"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.700905    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-systemd\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.700947    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.700954    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-run\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.700969    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-ovnkube-config\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.702926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.701001    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2c72dbe-b803-492b-9046-eeeacf69b9cf-cnibin\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.703909 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.701070    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2c72dbe-b803-492b-9046-eeeacf69b9cf-os-release\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.703909 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.701169    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-run-systemd\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.703909 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.701280    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-socket-dir\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.703909 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.701330    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-host-kubelet\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.703909 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.701398    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/25871790-7549-4c7b-ae62-759f32bf2c36-sys-fs\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.703909 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.701480    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-sys\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.703909 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.701536    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ec1b495c-bcb6-493b-811a-2e0957afa7a6-konnectivity-ca\") pod \"konnectivity-agent-j9b5n\" (UID: \"ec1b495c-bcb6-493b-811a-2e0957afa7a6\") " pod="kube-system/konnectivity-agent-j9b5n"
Jul 20 18:13:25.703909 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.701611    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-sysctl-conf\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.703909 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.701622    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-host\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.703909 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.702557    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-etc-tuned\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.703909 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.702586    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f2c72dbe-b803-492b-9046-eeeacf69b9cf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.704958 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.704411    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-ovn-node-metrics-cert\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.706476 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.706416    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg2ld\" (UniqueName: \"kubernetes.io/projected/d5a1608a-991c-4c92-978f-7f00e84fda46-kube-api-access-jg2ld\") pod \"node-ca-lkqb9\" (UID: \"d5a1608a-991c-4c92-978f-7f00e84fda46\") " pod="openshift-image-registry/node-ca-lkqb9"
Jul 20 18:13:25.706701 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.706652    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-tmp\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.710422 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.710365    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq75h\" (UniqueName: \"kubernetes.io/projected/25871790-7549-4c7b-ae62-759f32bf2c36-kube-api-access-wq75h\") pod \"aws-ebs-csi-driver-node-5jq27\" (UID: \"25871790-7549-4c7b-ae62-759f32bf2c36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.710969 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.710921    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8nlk\" (UniqueName: \"kubernetes.io/projected/80fea0b1-03fb-4254-be13-4315a5fc61cc-kube-api-access-b8nlk\") pod \"iptables-alerter-xdtrj\" (UID: \"80fea0b1-03fb-4254-be13-4315a5fc61cc\") " pod="openshift-network-operator/iptables-alerter-xdtrj"
Jul 20 18:13:25.711293 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.711249    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dc2z\" (UniqueName: \"kubernetes.io/projected/f2c72dbe-b803-492b-9046-eeeacf69b9cf-kube-api-access-6dc2z\") pod \"multus-additional-cni-plugins-pq7xn\" (UID: \"f2c72dbe-b803-492b-9046-eeeacf69b9cf\") " pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.711403 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.711336    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpdxr\" (UniqueName: \"kubernetes.io/projected/a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11-kube-api-access-zpdxr\") pod \"tuned-z242x\" (UID: \"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11\") " pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.713302 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.713259    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j492\" (UniqueName: \"kubernetes.io/projected/f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a-kube-api-access-2j492\") pod \"ovnkube-node-8f6m2\" (UID: \"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.760302 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.760258    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal" event={"ID":"873a0c6273e20da2e23695f53316d6b7","Type":"ContainerStarted","Data":"484a552f420edc6d6df42220b98afa88cb965b314ed6c0d33e7f6336ffb87a3b"}
Jul 20 18:13:25.796186 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796145    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/17e7c5f3-421c-4bdb-9e4e-67586149b43d-hosts-file\") pod \"node-resolver-8bf9p\" (UID: \"17e7c5f3-421c-4bdb-9e4e-67586149b43d\") " pod="openshift-dns/node-resolver-8bf9p"
Jul 20 18:13:25.796186 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796199    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17e7c5f3-421c-4bdb-9e4e-67586149b43d-tmp-dir\") pod \"node-resolver-8bf9p\" (UID: \"17e7c5f3-421c-4bdb-9e4e-67586149b43d\") " pod="openshift-dns/node-resolver-8bf9p"
Jul 20 18:13:25.796427 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-multus-cni-dir\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796427 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796247    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-multus-socket-dir-parent\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796427 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796306    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrr9b\" (UniqueName: \"kubernetes.io/projected/17e7c5f3-421c-4bdb-9e4e-67586149b43d-kube-api-access-jrr9b\") pod \"node-resolver-8bf9p\" (UID: \"17e7c5f3-421c-4bdb-9e4e-67586149b43d\") " pod="openshift-dns/node-resolver-8bf9p"
Jul 20 18:13:25.796427 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796327    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-multus-cni-dir\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796427 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796327    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-multus-socket-dir-parent\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796427 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796389    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/17e7c5f3-421c-4bdb-9e4e-67586149b43d-hosts-file\") pod \"node-resolver-8bf9p\" (UID: \"17e7c5f3-421c-4bdb-9e4e-67586149b43d\") " pod="openshift-dns/node-resolver-8bf9p"
Jul 20 18:13:25.796427 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796391    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-etc-kubernetes\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796427 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796428    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-etc-kubernetes\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796442    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs\") pod \"network-metrics-daemon-g4z7w\" (UID: \"1bae9e54-a7d1-46ba-9bb6-9602c8d53962\") " pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796470    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d755879-19d5-463b-bd98-83a890d00721-cni-binary-copy\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796495    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-run-k8s-cni-cncf-io\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796526    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-var-lib-cni-bin\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796551    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-var-lib-cni-multus\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796576    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-run-multus-certs\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796601    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-multus-conf-dir\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796616    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-run-k8s-cni-cncf-io\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796616    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-var-lib-cni-bin\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796623    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d755879-19d5-463b-bd98-83a890d00721-multus-daemon-config\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796662    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-var-lib-cni-multus\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796676    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmtrm\" (UniqueName: \"kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm\") pod \"network-check-target-hb8sv\" (UID: \"dd6d9efb-9a7b-4247-95c0-e8a3a56600eb\") " pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796703    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-run-multus-certs\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796710    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-cnibin\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796739    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-hostroot\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796767    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-run-netns\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796794    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-os-release\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.796822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796788    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-multus-conf-dir\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.797811 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796811    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-cnibin\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.797811 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:25.796577    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 18:13:25.797811 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796818    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-var-lib-kubelet\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.797811 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796849    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-var-lib-kubelet\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.797811 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796871    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncrt4\" (UniqueName: \"kubernetes.io/projected/6d755879-19d5-463b-bd98-83a890d00721-kube-api-access-ncrt4\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.797811 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796888    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-host-run-netns\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.797811 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:25.796938    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs podName:1bae9e54-a7d1-46ba-9bb6-9602c8d53962 nodeName:}" failed. No retries permitted until 2026-07-20 18:13:26.296905602 +0000 UTC m=+3.130877979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs") pod "network-metrics-daemon-g4z7w" (UID: "1bae9e54-a7d1-46ba-9bb6-9602c8d53962") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 18:13:25.797811 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796968    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-hostroot\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.797811 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796971    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-os-release\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.797811 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.796974    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-system-cni-dir\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.797811 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.797022    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d755879-19d5-463b-bd98-83a890d00721-system-cni-dir\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.797811 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.797041    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8fch\" (UniqueName: \"kubernetes.io/projected/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-kube-api-access-c8fch\") pod \"network-metrics-daemon-g4z7w\" (UID: \"1bae9e54-a7d1-46ba-9bb6-9602c8d53962\") " pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:25.797811 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.797329    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17e7c5f3-421c-4bdb-9e4e-67586149b43d-tmp-dir\") pod \"node-resolver-8bf9p\" (UID: \"17e7c5f3-421c-4bdb-9e4e-67586149b43d\") " pod="openshift-dns/node-resolver-8bf9p"
Jul 20 18:13:25.797811 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.797470    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d755879-19d5-463b-bd98-83a890d00721-multus-daemon-config\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.797811 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.797596    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d755879-19d5-463b-bd98-83a890d00721-cni-binary-copy\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.810219 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:25.810188    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 18:13:25.810219 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:25.810217    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 18:13:25.810417 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:25.810235    2569 projected.go:196] Error preparing data for projected volume kube-api-access-rmtrm for pod openshift-network-diagnostics/network-check-target-hb8sv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 18:13:25.810417 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:25.810302    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm podName:dd6d9efb-9a7b-4247-95c0-e8a3a56600eb nodeName:}" failed. No retries permitted until 2026-07-20 18:13:26.310284645 +0000 UTC m=+3.144257037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rmtrm" (UniqueName: "kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm") pod "network-check-target-hb8sv" (UID: "dd6d9efb-9a7b-4247-95c0-e8a3a56600eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 18:13:25.813732 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.813706    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrr9b\" (UniqueName: \"kubernetes.io/projected/17e7c5f3-421c-4bdb-9e4e-67586149b43d-kube-api-access-jrr9b\") pod \"node-resolver-8bf9p\" (UID: \"17e7c5f3-421c-4bdb-9e4e-67586149b43d\") " pod="openshift-dns/node-resolver-8bf9p"
Jul 20 18:13:25.814906 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.814860    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncrt4\" (UniqueName: \"kubernetes.io/projected/6d755879-19d5-463b-bd98-83a890d00721-kube-api-access-ncrt4\") pod \"multus-vc9wm\" (UID: \"6d755879-19d5-463b-bd98-83a890d00721\") " pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.815030 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.815009    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8fch\" (UniqueName: \"kubernetes.io/projected/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-kube-api-access-c8fch\") pod \"network-metrics-daemon-g4z7w\" (UID: \"1bae9e54-a7d1-46ba-9bb6-9602c8d53962\") " pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:25.880351 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.880313    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j9b5n"
Jul 20 18:13:25.895608 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.895574    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27"
Jul 20 18:13:25.906169 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.906139    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pq7xn"
Jul 20 18:13:25.912284 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.912263    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z242x"
Jul 20 18:13:25.921222 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.921195    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lkqb9"
Jul 20 18:13:25.938312 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.938283    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:25.948176 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.948151    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xdtrj"
Jul 20 18:13:25.956218 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.956187    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vc9wm"
Jul 20 18:13:25.964195 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:25.964174    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8bf9p"
Jul 20 18:13:26.200333 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:26.200251    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret\") pod \"global-pull-secret-syncer-pjhww\" (UID: \"31d6832c-6d75-41e8-b272-7edeb41be490\") " pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:26.200500 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:26.200381    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 18:13:26.200500 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:26.200445    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret podName:31d6832c-6d75-41e8-b272-7edeb41be490 nodeName:}" failed. No retries permitted until 2026-07-20 18:13:27.200430887 +0000 UTC m=+4.034403255 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret") pod "global-pull-secret-syncer-pjhww" (UID: "31d6832c-6d75-41e8-b272-7edeb41be490") : object "kube-system"/"original-pull-secret" not registered
Jul 20 18:13:26.301023 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:26.300987    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs\") pod \"network-metrics-daemon-g4z7w\" (UID: \"1bae9e54-a7d1-46ba-9bb6-9602c8d53962\") " pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:26.301205 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:26.301148    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 18:13:26.301260 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:26.301214    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs podName:1bae9e54-a7d1-46ba-9bb6-9602c8d53962 nodeName:}" failed. No retries permitted until 2026-07-20 18:13:27.301198056 +0000 UTC m=+4.135170423 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs") pod "network-metrics-daemon-g4z7w" (UID: "1bae9e54-a7d1-46ba-9bb6-9602c8d53962") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 18:13:26.401627 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:26.401582    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmtrm\" (UniqueName: \"kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm\") pod \"network-check-target-hb8sv\" (UID: \"dd6d9efb-9a7b-4247-95c0-e8a3a56600eb\") " pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:26.401816 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:26.401794    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 18:13:26.401876 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:26.401820    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 18:13:26.401876 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:26.401831    2569 projected.go:196] Error preparing data for projected volume kube-api-access-rmtrm for pod openshift-network-diagnostics/network-check-target-hb8sv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 18:13:26.401957 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:26.401900    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm podName:dd6d9efb-9a7b-4247-95c0-e8a3a56600eb nodeName:}" failed. No retries permitted until 2026-07-20 18:13:27.401878798 +0000 UTC m=+4.235851170 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmtrm" (UniqueName: "kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm") pod "network-check-target-hb8sv" (UID: "dd6d9efb-9a7b-4247-95c0-e8a3a56600eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 18:13:26.639486 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:26.639395    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 18:08:26 +0000 UTC" deadline="2028-02-05 06:29:12.452266313 +0000 UTC"
Jul 20 18:13:26.639486 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:26.639429    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13548h15m45.812841494s"
Jul 20 18:13:26.848639 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:26.848604    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25871790_7549_4c7b_ae62_759f32bf2c36.slice/crio-2b4a040a23b62b702f0919c011b66dac84f59d477fd134e638d1bca6aa143a22 WatchSource:0}: Error finding container 2b4a040a23b62b702f0919c011b66dac84f59d477fd134e638d1bca6aa143a22: Status 404 returned error can't find the container with id 2b4a040a23b62b702f0919c011b66dac84f59d477fd134e638d1bca6aa143a22
Jul 20 18:13:26.849256 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:26.849233    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6c44025_3fa9_4d6f_91a2_6c6dee0a7c11.slice/crio-4903226f98ef86cb593921830a6b2d546421224c8b1b8b85fbdab3857b9217b5 WatchSource:0}: Error finding container 4903226f98ef86cb593921830a6b2d546421224c8b1b8b85fbdab3857b9217b5: Status 404 returned error can't find the container with id 4903226f98ef86cb593921830a6b2d546421224c8b1b8b85fbdab3857b9217b5
Jul 20 18:13:26.851346 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:26.851198    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec1b495c_bcb6_493b_811a_2e0957afa7a6.slice/crio-9bf7ba5dbab88d8cb972abb0fcff894bf70b948bf84577a5fb476c98750349f9 WatchSource:0}: Error finding container 9bf7ba5dbab88d8cb972abb0fcff894bf70b948bf84577a5fb476c98750349f9: Status 404 returned error can't find the container with id 9bf7ba5dbab88d8cb972abb0fcff894bf70b948bf84577a5fb476c98750349f9
Jul 20 18:13:26.855827 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:26.855802    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80fea0b1_03fb_4254_be13_4315a5fc61cc.slice/crio-09463052066bbd66770fb088be05ce4c08a6f1ac569f84b743cb8821a9578316 WatchSource:0}: Error finding container 09463052066bbd66770fb088be05ce4c08a6f1ac569f84b743cb8821a9578316: Status 404 returned error can't find the container with id 09463052066bbd66770fb088be05ce4c08a6f1ac569f84b743cb8821a9578316
Jul 20 18:13:26.856678 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:26.856579    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d755879_19d5_463b_bd98_83a890d00721.slice/crio-fdad69c7a16d180c0f30b922b8f2e4620fdbdd22777d43040fbfc6c6807d3aef WatchSource:0}: Error finding container fdad69c7a16d180c0f30b922b8f2e4620fdbdd22777d43040fbfc6c6807d3aef: Status 404 returned error can't find the container with id fdad69c7a16d180c0f30b922b8f2e4620fdbdd22777d43040fbfc6c6807d3aef
Jul 20 18:13:26.858070 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:26.857993    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5a1608a_991c_4c92_978f_7f00e84fda46.slice/crio-5b126a273510a01efdebcee4577056b03e807eb76e6f7314ae5a15bae48c8b5a WatchSource:0}: Error finding container 5b126a273510a01efdebcee4577056b03e807eb76e6f7314ae5a15bae48c8b5a: Status 404 returned error can't find the container with id 5b126a273510a01efdebcee4577056b03e807eb76e6f7314ae5a15bae48c8b5a
Jul 20 18:13:26.858940 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:26.858916    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17e7c5f3_421c_4bdb_9e4e_67586149b43d.slice/crio-673083d1844d57cc0f41813a017d1a9f96cbb475e5f7b6e4a5d4b0f10350db43 WatchSource:0}: Error finding container 673083d1844d57cc0f41813a017d1a9f96cbb475e5f7b6e4a5d4b0f10350db43: Status 404 returned error can't find the container with id 673083d1844d57cc0f41813a017d1a9f96cbb475e5f7b6e4a5d4b0f10350db43
Jul 20 18:13:26.860353 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:26.860190    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2c72dbe_b803_492b_9046_eeeacf69b9cf.slice/crio-4ad66007431eff2249864a28b413d82ee88272c544cbba0bcd502cb729433f57 WatchSource:0}: Error finding container 4ad66007431eff2249864a28b413d82ee88272c544cbba0bcd502cb729433f57: Status 404 returned error can't find the container with id 4ad66007431eff2249864a28b413d82ee88272c544cbba0bcd502cb729433f57
Jul 20 18:13:26.860843 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:13:26.860706    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e6d59e_4e48_4cac_ada0_f6e8cf6cd91a.slice/crio-6b15c1386fdb9143c989a70f0d8fae9546d7b200ca3490c3daf0e66a4678e31e WatchSource:0}: Error finding container 6b15c1386fdb9143c989a70f0d8fae9546d7b200ca3490c3daf0e66a4678e31e: Status 404 returned error can't find the container with id 6b15c1386fdb9143c989a70f0d8fae9546d7b200ca3490c3daf0e66a4678e31e
Jul 20 18:13:27.209648 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.209242    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret\") pod \"global-pull-secret-syncer-pjhww\" (UID: \"31d6832c-6d75-41e8-b272-7edeb41be490\") " pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:27.209648 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:27.209411    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 18:13:27.209648 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:27.209605    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret podName:31d6832c-6d75-41e8-b272-7edeb41be490 nodeName:}" failed. No retries permitted until 2026-07-20 18:13:29.209584449 +0000 UTC m=+6.043556816 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret") pod "global-pull-secret-syncer-pjhww" (UID: "31d6832c-6d75-41e8-b272-7edeb41be490") : object "kube-system"/"original-pull-secret" not registered
Jul 20 18:13:27.310609 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.309920    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs\") pod \"network-metrics-daemon-g4z7w\" (UID: \"1bae9e54-a7d1-46ba-9bb6-9602c8d53962\") " pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:27.310609 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:27.310154    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 18:13:27.310609 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:27.310241    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs podName:1bae9e54-a7d1-46ba-9bb6-9602c8d53962 nodeName:}" failed. No retries permitted until 2026-07-20 18:13:29.310219516 +0000 UTC m=+6.144191918 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs") pod "network-metrics-daemon-g4z7w" (UID: "1bae9e54-a7d1-46ba-9bb6-9602c8d53962") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 18:13:27.410716 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.410663    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmtrm\" (UniqueName: \"kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm\") pod \"network-check-target-hb8sv\" (UID: \"dd6d9efb-9a7b-4247-95c0-e8a3a56600eb\") " pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:27.410896 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:27.410856    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 18:13:27.410896 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:27.410879    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 18:13:27.410896 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:27.410892    2569 projected.go:196] Error preparing data for projected volume kube-api-access-rmtrm for pod openshift-network-diagnostics/network-check-target-hb8sv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 18:13:27.411119 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:27.410954    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm podName:dd6d9efb-9a7b-4247-95c0-e8a3a56600eb nodeName:}" failed. No retries permitted until 2026-07-20 18:13:29.410934623 +0000 UTC m=+6.244907002 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmtrm" (UniqueName: "kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm") pod "network-check-target-hb8sv" (UID: "dd6d9efb-9a7b-4247-95c0-e8a3a56600eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 18:13:27.640563 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.640417    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 18:08:26 +0000 UTC" deadline="2028-04-08 05:34:05.823916244 +0000 UTC"
Jul 20 18:13:27.640563 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.640484    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15059h20m38.183436951s"
Jul 20 18:13:27.758150 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.754253    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:27.758150 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:27.754435    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:27.758150 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.754827    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:27.758150 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:27.754940    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:27.758150 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.755001    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:27.758150 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:27.755207    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:27.770572 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.770535    2569 generic.go:356] "Generic (PLEG): container finished" podID="873a0c6273e20da2e23695f53316d6b7" containerID="594bc8496db210fc3ba11aaa226407ca25c85c17f8f17aac85682d58b7161515" exitCode=0
Jul 20 18:13:27.770736 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.770626    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal" event={"ID":"873a0c6273e20da2e23695f53316d6b7","Type":"ContainerDied","Data":"594bc8496db210fc3ba11aaa226407ca25c85c17f8f17aac85682d58b7161515"}
Jul 20 18:13:27.784583 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.784550    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8bf9p" event={"ID":"17e7c5f3-421c-4bdb-9e4e-67586149b43d","Type":"ContainerStarted","Data":"673083d1844d57cc0f41813a017d1a9f96cbb475e5f7b6e4a5d4b0f10350db43"}
Jul 20 18:13:27.787735 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.787701    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lkqb9" event={"ID":"d5a1608a-991c-4c92-978f-7f00e84fda46","Type":"ContainerStarted","Data":"5b126a273510a01efdebcee4577056b03e807eb76e6f7314ae5a15bae48c8b5a"}
Jul 20 18:13:27.794347 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.794310    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z242x" event={"ID":"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11","Type":"ContainerStarted","Data":"4903226f98ef86cb593921830a6b2d546421224c8b1b8b85fbdab3857b9217b5"}
Jul 20 18:13:27.819764 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.819706    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-84.ec2.internal" event={"ID":"409747c0910991761a210f0528443183","Type":"ContainerStarted","Data":"02a17a04f9db70e5eb66797eb70313cedf90d35e965c9265177d4413ed191ab5"}
Jul 20 18:13:27.829114 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.829085    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2" event={"ID":"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a","Type":"ContainerStarted","Data":"6b15c1386fdb9143c989a70f0d8fae9546d7b200ca3490c3daf0e66a4678e31e"}
Jul 20 18:13:27.837200 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.837170    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vc9wm" event={"ID":"6d755879-19d5-463b-bd98-83a890d00721","Type":"ContainerStarted","Data":"fdad69c7a16d180c0f30b922b8f2e4620fdbdd22777d43040fbfc6c6807d3aef"}
Jul 20 18:13:27.845041 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.845010    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq7xn" event={"ID":"f2c72dbe-b803-492b-9046-eeeacf69b9cf","Type":"ContainerStarted","Data":"4ad66007431eff2249864a28b413d82ee88272c544cbba0bcd502cb729433f57"}
Jul 20 18:13:27.856389 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.856360    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xdtrj" event={"ID":"80fea0b1-03fb-4254-be13-4315a5fc61cc","Type":"ContainerStarted","Data":"09463052066bbd66770fb088be05ce4c08a6f1ac569f84b743cb8821a9578316"}
Jul 20 18:13:27.859456 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.859428    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j9b5n" event={"ID":"ec1b495c-bcb6-493b-811a-2e0957afa7a6","Type":"ContainerStarted","Data":"9bf7ba5dbab88d8cb972abb0fcff894bf70b948bf84577a5fb476c98750349f9"}
Jul 20 18:13:27.861691 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:27.861666    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27" event={"ID":"25871790-7549-4c7b-ae62-759f32bf2c36","Type":"ContainerStarted","Data":"2b4a040a23b62b702f0919c011b66dac84f59d477fd134e638d1bca6aa143a22"}
Jul 20 18:13:28.882225 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:28.882187    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal" event={"ID":"873a0c6273e20da2e23695f53316d6b7","Type":"ContainerStarted","Data":"e777e91e33ffea8bb9b61e7876667a15dca3aca2b43025335c1407ba83384028"}
Jul 20 18:13:28.897729 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:28.897670    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-84.ec2.internal" podStartSLOduration=3.89764943 podStartE2EDuration="3.89764943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 18:13:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:13:28.897199369 +0000 UTC m=+5.731171762" watchObservedRunningTime="2026-07-20 18:13:28.89764943 +0000 UTC m=+5.731621824"
Jul 20 18:13:28.898108 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:28.898066    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-84.ec2.internal" podStartSLOduration=3.898055865 podStartE2EDuration="3.898055865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 18:13:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:13:27.835252063 +0000 UTC m=+4.669224456" watchObservedRunningTime="2026-07-20 18:13:28.898055865 +0000 UTC m=+5.732028257"
Jul 20 18:13:29.228550 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:29.228174    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret\") pod \"global-pull-secret-syncer-pjhww\" (UID: \"31d6832c-6d75-41e8-b272-7edeb41be490\") " pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:29.228550 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:29.228345    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 18:13:29.228550 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:29.228433    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret podName:31d6832c-6d75-41e8-b272-7edeb41be490 nodeName:}" failed. No retries permitted until 2026-07-20 18:13:33.228410394 +0000 UTC m=+10.062382762 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret") pod "global-pull-secret-syncer-pjhww" (UID: "31d6832c-6d75-41e8-b272-7edeb41be490") : object "kube-system"/"original-pull-secret" not registered
Jul 20 18:13:29.328982 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:29.328947    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs\") pod \"network-metrics-daemon-g4z7w\" (UID: \"1bae9e54-a7d1-46ba-9bb6-9602c8d53962\") " pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:29.329315 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:29.329137    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 18:13:29.329315 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:29.329206    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs podName:1bae9e54-a7d1-46ba-9bb6-9602c8d53962 nodeName:}" failed. No retries permitted until 2026-07-20 18:13:33.32918643 +0000 UTC m=+10.163158801 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs") pod "network-metrics-daemon-g4z7w" (UID: "1bae9e54-a7d1-46ba-9bb6-9602c8d53962") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 18:13:29.430312 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:29.430275    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmtrm\" (UniqueName: \"kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm\") pod \"network-check-target-hb8sv\" (UID: \"dd6d9efb-9a7b-4247-95c0-e8a3a56600eb\") " pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:29.430504 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:29.430484    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 18:13:29.430573 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:29.430508    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 18:13:29.430573 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:29.430523    2569 projected.go:196] Error preparing data for projected volume kube-api-access-rmtrm for pod openshift-network-diagnostics/network-check-target-hb8sv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 18:13:29.430730 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:29.430583    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm podName:dd6d9efb-9a7b-4247-95c0-e8a3a56600eb nodeName:}" failed. No retries permitted until 2026-07-20 18:13:33.430561969 +0000 UTC m=+10.264534341 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmtrm" (UniqueName: "kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm") pod "network-check-target-hb8sv" (UID: "dd6d9efb-9a7b-4247-95c0-e8a3a56600eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 18:13:29.753827 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:29.753742    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:29.753827 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:29.753788    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:29.754105 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:29.753752    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:29.754105 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:29.753891    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:29.754105 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:29.753990    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:29.754269 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:29.754101    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:31.753636 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:31.753552    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:31.754108 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:31.753708    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:31.754195 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:31.754158    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:31.754195 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:31.754168    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:31.754307 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:31.754261    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:31.754374 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:31.754351    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:33.264191 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:33.264145    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret\") pod \"global-pull-secret-syncer-pjhww\" (UID: \"31d6832c-6d75-41e8-b272-7edeb41be490\") " pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:33.264693 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:33.264289    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 18:13:33.264693 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:33.264356    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret podName:31d6832c-6d75-41e8-b272-7edeb41be490 nodeName:}" failed. No retries permitted until 2026-07-20 18:13:41.26433821 +0000 UTC m=+18.098310580 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret") pod "global-pull-secret-syncer-pjhww" (UID: "31d6832c-6d75-41e8-b272-7edeb41be490") : object "kube-system"/"original-pull-secret" not registered
Jul 20 18:13:33.364759 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:33.364672    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs\") pod \"network-metrics-daemon-g4z7w\" (UID: \"1bae9e54-a7d1-46ba-9bb6-9602c8d53962\") " pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:33.364932 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:33.364828    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 18:13:33.364932 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:33.364915    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs podName:1bae9e54-a7d1-46ba-9bb6-9602c8d53962 nodeName:}" failed. No retries permitted until 2026-07-20 18:13:41.364896864 +0000 UTC m=+18.198869233 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs") pod "network-metrics-daemon-g4z7w" (UID: "1bae9e54-a7d1-46ba-9bb6-9602c8d53962") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 18:13:33.465793 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:33.465034    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmtrm\" (UniqueName: \"kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm\") pod \"network-check-target-hb8sv\" (UID: \"dd6d9efb-9a7b-4247-95c0-e8a3a56600eb\") " pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:33.465793 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:33.465298    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 18:13:33.465793 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:33.465326    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 18:13:33.465793 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:33.465342    2569 projected.go:196] Error preparing data for projected volume kube-api-access-rmtrm for pod openshift-network-diagnostics/network-check-target-hb8sv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 18:13:33.465793 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:33.465405    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm podName:dd6d9efb-9a7b-4247-95c0-e8a3a56600eb nodeName:}" failed. No retries permitted until 2026-07-20 18:13:41.465383686 +0000 UTC m=+18.299356060 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmtrm" (UniqueName: "kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm") pod "network-check-target-hb8sv" (UID: "dd6d9efb-9a7b-4247-95c0-e8a3a56600eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 18:13:33.754578 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:33.754491    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:33.754740 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:33.754594    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:33.754740 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:33.754624    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:33.754740 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:33.754705    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:33.755011 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:33.754974    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:33.755166 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:33.755032    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:35.753430 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:35.753394    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:35.753936 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:35.753545    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:35.754001 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:35.753986    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:35.754255 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:35.754221    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:35.754389 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:35.754325    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:35.754456 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:35.754413    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:36.900117 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:36.899396    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq7xn" event={"ID":"f2c72dbe-b803-492b-9046-eeeacf69b9cf","Type":"ContainerStarted","Data":"dc73caef2d20b76f7ce6323d24eef7915a426d99e2c980e58c22a10128dc9c36"}
Jul 20 18:13:36.907088 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:36.904563    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j9b5n" event={"ID":"ec1b495c-bcb6-493b-811a-2e0957afa7a6","Type":"ContainerStarted","Data":"831d04ef2c8da185e06107a49640392f8ac6ce55508b950cc16f4229466c95aa"}
Jul 20 18:13:36.910473 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:36.910441    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27" event={"ID":"25871790-7549-4c7b-ae62-759f32bf2c36","Type":"ContainerStarted","Data":"d7b8aa77b82c13e0ac38cbba2f9d83003cb54270e346b8af5cea78871233b13b"}
Jul 20 18:13:36.913429 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:36.913361    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8bf9p" event={"ID":"17e7c5f3-421c-4bdb-9e4e-67586149b43d","Type":"ContainerStarted","Data":"e8f9a732ac75c9c0a520f0beef57219717a83a6dd7f410ffe91de5d044c93030"}
Jul 20 18:13:36.915373 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:36.915347    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lkqb9" event={"ID":"d5a1608a-991c-4c92-978f-7f00e84fda46","Type":"ContainerStarted","Data":"ad5eb74ab16e722b0f0626324e202f5efa6e9827582d0cb3d7a694174f6b321e"}
Jul 20 18:13:36.918163 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:36.918011    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z242x" event={"ID":"a6c44025-3fa9-4d6f-91a2-6c6dee0a7c11","Type":"ContainerStarted","Data":"842b21ff9bd9db4a3463f11d38398998685ce6896915c35d23185cd04611e16f"}
Jul 20 18:13:36.955327 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:36.954541    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-z242x" podStartSLOduration=3.471011902 podStartE2EDuration="12.954519896s" totalImagesPullingTime="9.483507994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 18:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:13:36.954266822 +0000 UTC m=+13.788239211" watchObservedRunningTime="2026-07-20 18:13:36.954519896 +0000 UTC m=+13.788492285"
Jul 20 18:13:36.955327 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:36.955070    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-j9b5n" podStartSLOduration=3.705691548 podStartE2EDuration="12.955037322s" totalImagesPullingTime="9.249345774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:13:36.938006051 +0000 UTC m=+13.771978439" watchObservedRunningTime="2026-07-20 18:13:36.955037322 +0000 UTC m=+13.789009712"
Jul 20 18:13:36.968562 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:36.968175    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lkqb9" podStartSLOduration=3.512995017 podStartE2EDuration="12.968156402s" totalImagesPullingTime="9.455161385s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 18:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:13:36.967967757 +0000 UTC m=+13.801940159" watchObservedRunningTime="2026-07-20 18:13:36.968156402 +0000 UTC m=+13.802128791"
Jul 20 18:13:36.989084 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:36.989016    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8bf9p" podStartSLOduration=3.5341916920000003 podStartE2EDuration="12.988995586s" totalImagesPullingTime="9.454803894s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 18:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:13:36.988265656 +0000 UTC m=+13.822238063" watchObservedRunningTime="2026-07-20 18:13:36.988995586 +0000 UTC m=+13.822967975"
Jul 20 18:13:37.753519 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:37.753204    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:37.753676 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:37.753580    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:37.753676 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:37.753414    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:37.753676 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:37.753224    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:37.753829 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:37.753677    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:37.753829 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:37.753751    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:38.924710 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:38.924665    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xdtrj" event={"ID":"80fea0b1-03fb-4254-be13-4315a5fc61cc","Type":"ContainerStarted","Data":"cbc2c1dc0dd63189f2cfe82c0475208548929c61b729f59173366a2a71c9ea32"}
Jul 20 18:13:38.939489 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:38.939432    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xdtrj" podStartSLOduration=5.481254965 podStartE2EDuration="14.939414731s" totalImagesPullingTime="9.458159766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 18:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:13:38.939145412 +0000 UTC m=+15.773117801" watchObservedRunningTime="2026-07-20 18:13:38.939414731 +0000 UTC m=+15.773387118"
Jul 20 18:13:39.753101 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:39.753067    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:39.753333 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:39.753071    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:39.753333 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:39.753191    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:39.753333 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:39.753291    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:39.753333 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:39.753079    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:39.753546 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:39.753376    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:40.929527 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:40.929487    2569 generic.go:356] "Generic (PLEG): container finished" podID="f2c72dbe-b803-492b-9046-eeeacf69b9cf" containerID="dc73caef2d20b76f7ce6323d24eef7915a426d99e2c980e58c22a10128dc9c36" exitCode=0
Jul 20 18:13:40.930177 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:40.929540    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq7xn" event={"ID":"f2c72dbe-b803-492b-9046-eeeacf69b9cf","Type":"ContainerDied","Data":"dc73caef2d20b76f7ce6323d24eef7915a426d99e2c980e58c22a10128dc9c36"}
Jul 20 18:13:41.128803 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:41.128732    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-j9b5n"
Jul 20 18:13:41.129443 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:41.129422    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-j9b5n"
Jul 20 18:13:41.328075 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:41.327999    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret\") pod \"global-pull-secret-syncer-pjhww\" (UID: \"31d6832c-6d75-41e8-b272-7edeb41be490\") " pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:41.328251 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:41.328163    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 18:13:41.328251 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:41.328239    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret podName:31d6832c-6d75-41e8-b272-7edeb41be490 nodeName:}" failed. No retries permitted until 2026-07-20 18:13:57.328222435 +0000 UTC m=+34.162194804 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret") pod "global-pull-secret-syncer-pjhww" (UID: "31d6832c-6d75-41e8-b272-7edeb41be490") : object "kube-system"/"original-pull-secret" not registered
Jul 20 18:13:41.429330 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:41.429249    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs\") pod \"network-metrics-daemon-g4z7w\" (UID: \"1bae9e54-a7d1-46ba-9bb6-9602c8d53962\") " pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:41.429470 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:41.429416    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 18:13:41.429519 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:41.429485    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs podName:1bae9e54-a7d1-46ba-9bb6-9602c8d53962 nodeName:}" failed. No retries permitted until 2026-07-20 18:13:57.429466628 +0000 UTC m=+34.263438994 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs") pod "network-metrics-daemon-g4z7w" (UID: "1bae9e54-a7d1-46ba-9bb6-9602c8d53962") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 18:13:41.529860 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:41.529822    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmtrm\" (UniqueName: \"kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm\") pod \"network-check-target-hb8sv\" (UID: \"dd6d9efb-9a7b-4247-95c0-e8a3a56600eb\") " pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:41.530042 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:41.530012    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 18:13:41.530042 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:41.530042    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 18:13:41.530217 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:41.530068    2569 projected.go:196] Error preparing data for projected volume kube-api-access-rmtrm for pod openshift-network-diagnostics/network-check-target-hb8sv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 18:13:41.530217 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:41.530128    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm podName:dd6d9efb-9a7b-4247-95c0-e8a3a56600eb nodeName:}" failed. No retries permitted until 2026-07-20 18:13:57.530113032 +0000 UTC m=+34.364085410 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmtrm" (UniqueName: "kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm") pod "network-check-target-hb8sv" (UID: "dd6d9efb-9a7b-4247-95c0-e8a3a56600eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 18:13:41.753321 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:41.753250    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:41.753321 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:41.753284    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:41.753502 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:41.753250    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:41.753502 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:41.753351    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:41.753502 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:41.753451    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:41.753643 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:41.753503    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:43.754063 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:43.754010    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:43.754497 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:43.754135    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:43.754497 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:43.754182    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:43.754497 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:43.754178    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:43.754497 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:43.754260    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:43.754497 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:43.754343    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:45.753109 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:45.753065    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:45.753109 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:45.753098    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:45.753657 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:45.753064    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:45.753657 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:45.753207    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:45.753657 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:45.753267    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:45.753657 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:45.753364    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:45.942440 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:45.942186    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vc9wm" event={"ID":"6d755879-19d5-463b-bd98-83a890d00721","Type":"ContainerStarted","Data":"86dcaee2789de3286ad609d0e4cece69a2d04c89a25815f43c25f1d8a6972728"}
Jul 20 18:13:45.961647 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:45.961599    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vc9wm" podStartSLOduration=2.979801551 podStartE2EDuration="21.961583167s" totalImagesPullingTime="18.981781616s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 18:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:13:45.96152076 +0000 UTC m=+22.795493149" watchObservedRunningTime="2026-07-20 18:13:45.961583167 +0000 UTC m=+22.795555555"
Jul 20 18:13:46.698357 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:46.698335    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 20 18:13:46.946160 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:46.945919    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27" event={"ID":"25871790-7549-4c7b-ae62-759f32bf2c36","Type":"ContainerStarted","Data":"5a299c92debf068771c45c26b3297aed1ae0d894ff1b26770e51c321565e7a03"}
Jul 20 18:13:46.948995 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:46.948974    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2" event={"ID":"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a","Type":"ContainerStarted","Data":"23475fa9b9f019cb87b74af2a5072670a30e71333bbf1a57ce06204adca2d735"}
Jul 20 18:13:46.949111 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:46.949032    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2" event={"ID":"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a","Type":"ContainerStarted","Data":"7cc42ed7b537ed42cb83604df27cb22baf00adb05079b72700a3e5ac657dc87f"}
Jul 20 18:13:46.949111 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:46.949087    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2" event={"ID":"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a","Type":"ContainerStarted","Data":"25739e629b51d2857ddf6eca92c6a0d3cae4f327352a635db0882ac17d5a8706"}
Jul 20 18:13:46.949185 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:46.949118    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2" event={"ID":"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a","Type":"ContainerStarted","Data":"4ce181a04cb8c0aace1a4ec588570cbb74ddf970ee466a50c934cd9ba0af9636"}
Jul 20 18:13:47.640935 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:47.640786    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T18:13:46.698352531Z","UUID":"955b809c-9d0a-4d7c-919e-d8607f88e7cd","Handler":null,"Name":"","Endpoint":""}
Jul 20 18:13:47.642506 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:47.642476    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 20 18:13:47.642506 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:47.642511    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 20 18:13:47.752820 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:47.752789    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:47.752820 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:47.752812    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:47.753080 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:47.752831    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:47.753080 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:47.752924    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:47.753193 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:47.753070    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:47.753193 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:47.753163    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:47.955651 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:47.955576    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2" event={"ID":"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a","Type":"ContainerStarted","Data":"46c42e8134be428682825dccde0db0a6a1b17cfd2073b29af7320e7f1c7d23f5"}
Jul 20 18:13:47.956099 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:47.955665    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2" event={"ID":"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a","Type":"ContainerStarted","Data":"10ff0d8ded1feffe4592d56468bf310bebb3035bc537fbc256a484c3c4b35a43"}
Jul 20 18:13:48.959345 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:48.959297    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27" event={"ID":"25871790-7549-4c7b-ae62-759f32bf2c36","Type":"ContainerStarted","Data":"ff5b70f899e763486082e15fd44240ff8e57a03dc5f127eded3e6371a274667f"}
Jul 20 18:13:48.977246 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:48.977187    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5jq27" podStartSLOduration=4.185047554 podStartE2EDuration="24.977173477s" totalImagesPullingTime="20.792125923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 18:13:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:13:48.976737464 +0000 UTC m=+25.810709853" watchObservedRunningTime="2026-07-20 18:13:48.977173477 +0000 UTC m=+25.811145876"
Jul 20 18:13:49.753292 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:49.753262    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:49.753470 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:49.753262    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:49.753470 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:49.753390    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:49.753470 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:49.753415    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:49.753636 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:49.753538    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:49.753683 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:49.753629    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:49.948662 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:49.948623    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-j9b5n"
Jul 20 18:13:49.948871 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:49.948747    2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 20 18:13:49.949338 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:49.949293    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-j9b5n"
Jul 20 18:13:49.965768 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:49.965729    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2" event={"ID":"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a","Type":"ContainerStarted","Data":"56b24815a8c46e53642ee84a522178832189b0f223c0cae1da057e1cad86e32c"}
Jul 20 18:13:51.753003 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:51.752968    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:51.753451 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:51.753018    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:51.753451 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:51.752969    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:51.753451 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:51.753131    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:51.753451 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:51.753238    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:51.753451 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:51.753350    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:52.975873 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:52.975666    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2" event={"ID":"f0e6d59e-4e48-4cac-ada0-f6e8cf6cd91a","Type":"ContainerStarted","Data":"81dd6da8e045842280c7a20ec29250c25049353df7cd239d9e46bc72b339ca47"}
Jul 20 18:13:52.976480 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:52.975926    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:52.978099 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:52.978074    2569 generic.go:356] "Generic (PLEG): container finished" podID="f2c72dbe-b803-492b-9046-eeeacf69b9cf" containerID="09d8c8ec259f9ddc1320adcf334a3025825a2fe71daea73a11a21d321d0a6a51" exitCode=0
Jul 20 18:13:52.978208 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:52.978120    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq7xn" event={"ID":"f2c72dbe-b803-492b-9046-eeeacf69b9cf","Type":"ContainerDied","Data":"09d8c8ec259f9ddc1320adcf334a3025825a2fe71daea73a11a21d321d0a6a51"}
Jul 20 18:13:52.991803 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:52.991784    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:53.007443 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:53.007400    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2" podStartSLOduration=9.559230821 podStartE2EDuration="29.007388423s" totalImagesPullingTime="19.448157602s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 18:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:13:53.007292887 +0000 UTC m=+29.841265298" watchObservedRunningTime="2026-07-20 18:13:53.007388423 +0000 UTC m=+29.841360811"
Jul 20 18:13:53.755938 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:53.755908    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:53.756122 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:53.756008    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:53.756122 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:53.756023    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:53.756240 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:53.756118    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:53.756240 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:53.756164    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:53.756240 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:53.756213    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:53.980539 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:53.980517    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:53.980858 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:53.980582    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:53.997249 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:53.997229    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:13:54.176033 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:54.176007    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pjhww"]
Jul 20 18:13:54.176169 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:54.176118    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:54.176224 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:54.176206    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:54.180096 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:54.180049    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g4z7w"]
Jul 20 18:13:54.180231 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:54.180168    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:54.180295 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:54.180275    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:54.180591 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:54.180567    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hb8sv"]
Jul 20 18:13:54.180699 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:54.180653    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:54.180782 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:54.180749    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:54.984565 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:54.984532    2569 generic.go:356] "Generic (PLEG): container finished" podID="f2c72dbe-b803-492b-9046-eeeacf69b9cf" containerID="4ec406d8555e1907a838ac45c54945d2c95c7cdacfe5e63dedc51a427eecbc23" exitCode=0
Jul 20 18:13:54.984946 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:54.984612    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq7xn" event={"ID":"f2c72dbe-b803-492b-9046-eeeacf69b9cf","Type":"ContainerDied","Data":"4ec406d8555e1907a838ac45c54945d2c95c7cdacfe5e63dedc51a427eecbc23"}
Jul 20 18:13:55.752852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:55.752819    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:55.752852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:55.752831    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:55.753015 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:55.752929    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:55.753015 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:55.752981    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:55.753015 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:55.752992    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:55.753145 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:55.753075    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:56.990741 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:56.990578    2569 generic.go:356] "Generic (PLEG): container finished" podID="f2c72dbe-b803-492b-9046-eeeacf69b9cf" containerID="bd58a847d5408d50cb87f05ea518f49709815d3cba38912b2d2805336f74b246" exitCode=0
Jul 20 18:13:56.991154 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:56.990647    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq7xn" event={"ID":"f2c72dbe-b803-492b-9046-eeeacf69b9cf","Type":"ContainerDied","Data":"bd58a847d5408d50cb87f05ea518f49709815d3cba38912b2d2805336f74b246"}
Jul 20 18:13:57.350530 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:57.350502    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret\") pod \"global-pull-secret-syncer-pjhww\" (UID: \"31d6832c-6d75-41e8-b272-7edeb41be490\") " pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:57.350744 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:57.350613    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 18:13:57.350744 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:57.350669    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret podName:31d6832c-6d75-41e8-b272-7edeb41be490 nodeName:}" failed. No retries permitted until 2026-07-20 18:14:29.350652506 +0000 UTC m=+66.184624878 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret") pod "global-pull-secret-syncer-pjhww" (UID: "31d6832c-6d75-41e8-b272-7edeb41be490") : object "kube-system"/"original-pull-secret" not registered
Jul 20 18:13:57.451652 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:57.451622    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs\") pod \"network-metrics-daemon-g4z7w\" (UID: \"1bae9e54-a7d1-46ba-9bb6-9602c8d53962\") " pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:57.451795 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:57.451741    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 18:13:57.451795 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:57.451787    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs podName:1bae9e54-a7d1-46ba-9bb6-9602c8d53962 nodeName:}" failed. No retries permitted until 2026-07-20 18:14:29.451775279 +0000 UTC m=+66.285747646 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs") pod "network-metrics-daemon-g4z7w" (UID: "1bae9e54-a7d1-46ba-9bb6-9602c8d53962") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 18:13:57.552766 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:57.552733    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmtrm\" (UniqueName: \"kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm\") pod \"network-check-target-hb8sv\" (UID: \"dd6d9efb-9a7b-4247-95c0-e8a3a56600eb\") " pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:57.552917 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:57.552854    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 18:13:57.552917 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:57.552872    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 18:13:57.552917 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:57.552880    2569 projected.go:196] Error preparing data for projected volume kube-api-access-rmtrm for pod openshift-network-diagnostics/network-check-target-hb8sv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 18:13:57.553014 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:57.552930    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm podName:dd6d9efb-9a7b-4247-95c0-e8a3a56600eb nodeName:}" failed. No retries permitted until 2026-07-20 18:14:29.552914256 +0000 UTC m=+66.386886624 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmtrm" (UniqueName: "kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm") pod "network-check-target-hb8sv" (UID: "dd6d9efb-9a7b-4247-95c0-e8a3a56600eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 18:13:57.752785 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:57.752755    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:57.752958 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:57.752755    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:57.752958 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:57.752874    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjhww" podUID="31d6832c-6d75-41e8-b272-7edeb41be490"
Jul 20 18:13:57.752958 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:57.752912    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hb8sv" podUID="dd6d9efb-9a7b-4247-95c0-e8a3a56600eb"
Jul 20 18:13:57.752958 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:57.752755    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:57.753202 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:57.753003    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:13:59.505872 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.505841    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-84.ec2.internal" event="NodeReady"
Jul 20 18:13:59.506406 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.506021    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 20 18:13:59.538176 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.538137    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5875d48c7b-kl9md"]
Jul 20 18:13:59.559909 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.559862    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-85nj6"]
Jul 20 18:13:59.560092 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.559995    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.562444 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.562422    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 20 18:13:59.562553 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.562450    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pzczt\""
Jul 20 18:13:59.562714 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.562699    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 20 18:13:59.562817 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.562800    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 20 18:13:59.574147 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.574131    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 20 18:13:59.575864 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.575848    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5n5pt"]
Jul 20 18:13:59.575994 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.575974    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-85nj6"
Jul 20 18:13:59.577989 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.577974    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 20 18:13:59.578102 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.578006    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 20 18:13:59.578102 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.578041    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wr2zv\""
Jul 20 18:13:59.594205 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.594184    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5875d48c7b-kl9md"]
Jul 20 18:13:59.594291 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.594212    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-85nj6"]
Jul 20 18:13:59.594291 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.594236    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5n5pt"]
Jul 20 18:13:59.594368 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.594292    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5n5pt"
Jul 20 18:13:59.596923 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.596907    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 20 18:13:59.597022 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.596959    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 20 18:13:59.597022 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.596984    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 20 18:13:59.597118 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.597023    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9f5rk\""
Jul 20 18:13:59.666665 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.666630    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-image-registry-private-configuration\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.666665 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.666675    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-certificates\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.666885 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.666704    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67wsr\" (UniqueName: \"kubernetes.io/projected/d223e041-067c-49a8-a19c-b4879963bc0e-kube-api-access-67wsr\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:13:59.666885 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.666781    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-installation-pull-secrets\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.666885 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.666833    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d223e041-067c-49a8-a19c-b4879963bc0e-tmp-dir\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:13:59.666885 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.666856    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:13:59.667120 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.666896    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-bound-sa-token\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.667120 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.666920    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-trusted-ca\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.667120 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.666946    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn964\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-kube-api-access-mn964\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.667120 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.666970    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d223e041-067c-49a8-a19c-b4879963bc0e-config-volume\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:13:59.667120 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.667014    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.667120 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.667086    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-ca-trust-extracted\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.752914 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.752887    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:13:59.753106 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.752887    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:13:59.753180 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.752887    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:13:59.755715 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.755692    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 18:13:59.755849 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.755752    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ls8qp\""
Jul 20 18:13:59.755849 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.755776    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n7qbr\""
Jul 20 18:13:59.755984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.755949    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 18:13:59.755984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.755956    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 18:13:59.755984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.755970    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 18:13:59.767989 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.767963    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-image-registry-private-configuration\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.768123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.768002    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-certificates\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.768123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.768027    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67wsr\" (UniqueName: \"kubernetes.io/projected/d223e041-067c-49a8-a19c-b4879963bc0e-kube-api-access-67wsr\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:13:59.768123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.768106    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-installation-pull-secrets\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.768280 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.768169    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d223e041-067c-49a8-a19c-b4879963bc0e-tmp-dir\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:13:59.768280 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.768220    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:13:59.768280 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.768251    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvtz2\" (UniqueName: \"kubernetes.io/projected/fb74a816-b89d-4e4d-b322-cc029854d831-kube-api-access-hvtz2\") pod \"ingress-canary-5n5pt\" (UID: \"fb74a816-b89d-4e4d-b322-cc029854d831\") " pod="openshift-ingress-canary/ingress-canary-5n5pt"
Jul 20 18:13:59.768427 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.768294    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-bound-sa-token\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.768427 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.768324    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-trusted-ca\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.768427 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:59.768348    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 18:13:59.768427 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.768353    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn964\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-kube-api-access-mn964\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.768427 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.768376    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d223e041-067c-49a8-a19c-b4879963bc0e-config-volume\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:13:59.768427 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:59.768414    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls podName:d223e041-067c-49a8-a19c-b4879963bc0e nodeName:}" failed. No retries permitted until 2026-07-20 18:14:00.268394254 +0000 UTC m=+37.102366623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls") pod "dns-default-85nj6" (UID: "d223e041-067c-49a8-a19c-b4879963bc0e") : secret "dns-default-metrics-tls" not found
Jul 20 18:13:59.768687 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.768667    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-certificates\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.769307 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.769283    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d223e041-067c-49a8-a19c-b4879963bc0e-config-volume\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:13:59.769442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.769419    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.769504 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.769483    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb74a816-b89d-4e4d-b322-cc029854d831-cert\") pod \"ingress-canary-5n5pt\" (UID: \"fb74a816-b89d-4e4d-b322-cc029854d831\") " pod="openshift-ingress-canary/ingress-canary-5n5pt"
Jul 20 18:13:59.769585 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.769569    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-ca-trust-extracted\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.769732 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:59.769711    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 18:13:59.769795 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:59.769734    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5875d48c7b-kl9md: secret "image-registry-tls" not found
Jul 20 18:13:59.769795 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:13:59.769775    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls podName:6fe2898d-c51c-4fe4-a2c5-c554e9e499be nodeName:}" failed. No retries permitted until 2026-07-20 18:14:00.269761256 +0000 UTC m=+37.103733635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls") pod "image-registry-5875d48c7b-kl9md" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be") : secret "image-registry-tls" not found
Jul 20 18:13:59.769915 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.769809    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d223e041-067c-49a8-a19c-b4879963bc0e-tmp-dir\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:13:59.769915 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.769855    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-ca-trust-extracted\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.771432 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.771411    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-trusted-ca\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.773342 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.773322    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-image-registry-private-configuration\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.773443 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.773342    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-installation-pull-secrets\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.776316 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.776298    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67wsr\" (UniqueName: \"kubernetes.io/projected/d223e041-067c-49a8-a19c-b4879963bc0e-kube-api-access-67wsr\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:13:59.784013 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.783988    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn964\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-kube-api-access-mn964\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.784121 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.784048    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-bound-sa-token\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:13:59.870777 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.870752    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb74a816-b89d-4e4d-b322-cc029854d831-cert\") pod \"ingress-canary-5n5pt\" (UID: \"fb74a816-b89d-4e4d-b322-cc029854d831\") " pod="openshift-ingress-canary/ingress-canary-5n5pt"
Jul 20 18:13:59.870906 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.870813    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvtz2\" (UniqueName: \"kubernetes.io/projected/fb74a816-b89d-4e4d-b322-cc029854d831-kube-api-access-hvtz2\") pod \"ingress-canary-5n5pt\" (UID: \"fb74a816-b89d-4e4d-b322-cc029854d831\") " pod="openshift-ingress-canary/ingress-canary-5n5pt"
Jul 20 18:13:59.873091 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.873071    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb74a816-b89d-4e4d-b322-cc029854d831-cert\") pod \"ingress-canary-5n5pt\" (UID: \"fb74a816-b89d-4e4d-b322-cc029854d831\") " pod="openshift-ingress-canary/ingress-canary-5n5pt"
Jul 20 18:13:59.878827 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.878805    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvtz2\" (UniqueName: \"kubernetes.io/projected/fb74a816-b89d-4e4d-b322-cc029854d831-kube-api-access-hvtz2\") pod \"ingress-canary-5n5pt\" (UID: \"fb74a816-b89d-4e4d-b322-cc029854d831\") " pod="openshift-ingress-canary/ingress-canary-5n5pt"
Jul 20 18:13:59.902792 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:13:59.902770    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5n5pt"
Jul 20 18:14:00.131445 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:00.131413    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5n5pt"]
Jul 20 18:14:00.136559 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:14:00.136518    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb74a816_b89d_4e4d_b322_cc029854d831.slice/crio-4496900ba95ef0db166d65bf9e1e184ea4010e6daca764033de635ff9c0295f3 WatchSource:0}: Error finding container 4496900ba95ef0db166d65bf9e1e184ea4010e6daca764033de635ff9c0295f3: Status 404 returned error can't find the container with id 4496900ba95ef0db166d65bf9e1e184ea4010e6daca764033de635ff9c0295f3
Jul 20 18:14:00.273581 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:00.273543    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:14:00.273743 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:00.273613    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:14:00.273743 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:00.273701    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 18:14:00.273743 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:00.273711    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 18:14:00.273743 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:00.273720    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5875d48c7b-kl9md: secret "image-registry-tls" not found
Jul 20 18:14:00.273863 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:00.273762    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls podName:d223e041-067c-49a8-a19c-b4879963bc0e nodeName:}" failed. No retries permitted until 2026-07-20 18:14:01.27375003 +0000 UTC m=+38.107722396 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls") pod "dns-default-85nj6" (UID: "d223e041-067c-49a8-a19c-b4879963bc0e") : secret "dns-default-metrics-tls" not found
Jul 20 18:14:00.273863 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:00.273775    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls podName:6fe2898d-c51c-4fe4-a2c5-c554e9e499be nodeName:}" failed. No retries permitted until 2026-07-20 18:14:01.273769204 +0000 UTC m=+38.107741570 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls") pod "image-registry-5875d48c7b-kl9md" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be") : secret "image-registry-tls" not found
Jul 20 18:14:00.999686 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:00.999651    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5n5pt" event={"ID":"fb74a816-b89d-4e4d-b322-cc029854d831","Type":"ContainerStarted","Data":"4496900ba95ef0db166d65bf9e1e184ea4010e6daca764033de635ff9c0295f3"}
Jul 20 18:14:01.281025 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:01.280938    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:14:01.281025 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:01.281000    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:14:01.281239 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:01.281110    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 18:14:01.281239 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:01.281129    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 18:14:01.281239 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:01.281140    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5875d48c7b-kl9md: secret "image-registry-tls" not found
Jul 20 18:14:01.281239 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:01.281174    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls podName:d223e041-067c-49a8-a19c-b4879963bc0e nodeName:}" failed. No retries permitted until 2026-07-20 18:14:03.281159628 +0000 UTC m=+40.115131995 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls") pod "dns-default-85nj6" (UID: "d223e041-067c-49a8-a19c-b4879963bc0e") : secret "dns-default-metrics-tls" not found
Jul 20 18:14:01.281239 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:01.281188    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls podName:6fe2898d-c51c-4fe4-a2c5-c554e9e499be nodeName:}" failed. No retries permitted until 2026-07-20 18:14:03.281182107 +0000 UTC m=+40.115154472 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls") pod "image-registry-5875d48c7b-kl9md" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be") : secret "image-registry-tls" not found
Jul 20 18:14:03.297131 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:03.296848    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:14:03.297597 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:03.297191    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:14:03.297597 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:03.297020    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 18:14:03.297597 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:03.297301    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls podName:d223e041-067c-49a8-a19c-b4879963bc0e nodeName:}" failed. No retries permitted until 2026-07-20 18:14:07.297281229 +0000 UTC m=+44.131253598 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls") pod "dns-default-85nj6" (UID: "d223e041-067c-49a8-a19c-b4879963bc0e") : secret "dns-default-metrics-tls" not found
Jul 20 18:14:03.297597 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:03.297322    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 18:14:03.297597 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:03.297341    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5875d48c7b-kl9md: secret "image-registry-tls" not found
Jul 20 18:14:03.297597 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:03.297403    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls podName:6fe2898d-c51c-4fe4-a2c5-c554e9e499be nodeName:}" failed. No retries permitted until 2026-07-20 18:14:07.297384655 +0000 UTC m=+44.131357022 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls") pod "image-registry-5875d48c7b-kl9md" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be") : secret "image-registry-tls" not found
Jul 20 18:14:03.682161 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:03.682069    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:03.773272 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:03.773237    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:05.011288 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:05.011251    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5n5pt" event={"ID":"fb74a816-b89d-4e4d-b322-cc029854d831","Type":"ContainerStarted","Data":"c0399fa4cdab1a3c65198223840da90846641743dfcb5d8ea11d5f87c60ec5ec"}
Jul 20 18:14:05.014627 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:05.014593    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq7xn" event={"ID":"f2c72dbe-b803-492b-9046-eeeacf69b9cf","Type":"ContainerStarted","Data":"1d9ea3bfa35d5fbeb40cb2b14364654586112965b700bf10050827f11488642d"}
Jul 20 18:14:05.027378 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:05.027297    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5n5pt" podStartSLOduration=1.383587458 podStartE2EDuration="6.027282355s" totalImagesPullingTime="4.643694897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:13:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:14:05.025671975 +0000 UTC m=+41.859644363" watchObservedRunningTime="2026-07-20 18:14:05.027282355 +0000 UTC m=+41.861254743"
Jul 20 18:14:06.019095 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:06.019041    2569 generic.go:356] "Generic (PLEG): container finished" podID="f2c72dbe-b803-492b-9046-eeeacf69b9cf" containerID="1d9ea3bfa35d5fbeb40cb2b14364654586112965b700bf10050827f11488642d" exitCode=0
Jul 20 18:14:06.019095 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:06.019087    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq7xn" event={"ID":"f2c72dbe-b803-492b-9046-eeeacf69b9cf","Type":"ContainerDied","Data":"1d9ea3bfa35d5fbeb40cb2b14364654586112965b700bf10050827f11488642d"}
Jul 20 18:14:06.307521 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:06.307437    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:06.361127 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:06.361097    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:07.006573 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.006539    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"]
Jul 20 18:14:07.023651 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.023623    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"]
Jul 20 18:14:07.024039 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.023712    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:14:07.025368 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.025344    2569 generic.go:356] "Generic (PLEG): container finished" podID="f2c72dbe-b803-492b-9046-eeeacf69b9cf" containerID="6bec76082bda75fd95fcf9496e5526704d7bb973d300c3c4720fd6b5eb917d2a" exitCode=0
Jul 20 18:14:07.025478 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.025388    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq7xn" event={"ID":"f2c72dbe-b803-492b-9046-eeeacf69b9cf","Type":"ContainerDied","Data":"6bec76082bda75fd95fcf9496e5526704d7bb973d300c3c4720fd6b5eb917d2a"}
Jul 20 18:14:07.026893 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.026863    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 20 18:14:07.026893 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.026871    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 20 18:14:07.027067 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.026927    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 20 18:14:07.027067 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.026929    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 20 18:14:07.126727 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.124342    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5c78\" (UniqueName: \"kubernetes.io/projected/ec6b7d20-2ce7-4819-82bf-36a12f08275b-kube-api-access-q5c78\") pod \"klusterlet-addon-workmgr-98b95c4fd-srl79\" (UID: \"ec6b7d20-2ce7-4819-82bf-36a12f08275b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:14:07.126727 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.124681    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec6b7d20-2ce7-4819-82bf-36a12f08275b-tmp\") pod \"klusterlet-addon-workmgr-98b95c4fd-srl79\" (UID: \"ec6b7d20-2ce7-4819-82bf-36a12f08275b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:14:07.126727 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.124822    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ec6b7d20-2ce7-4819-82bf-36a12f08275b-klusterlet-config\") pod \"klusterlet-addon-workmgr-98b95c4fd-srl79\" (UID: \"ec6b7d20-2ce7-4819-82bf-36a12f08275b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:14:07.225921 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.225873    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ec6b7d20-2ce7-4819-82bf-36a12f08275b-klusterlet-config\") pod \"klusterlet-addon-workmgr-98b95c4fd-srl79\" (UID: \"ec6b7d20-2ce7-4819-82bf-36a12f08275b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:14:07.226121 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.225938    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5c78\" (UniqueName: \"kubernetes.io/projected/ec6b7d20-2ce7-4819-82bf-36a12f08275b-kube-api-access-q5c78\") pod \"klusterlet-addon-workmgr-98b95c4fd-srl79\" (UID: \"ec6b7d20-2ce7-4819-82bf-36a12f08275b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:14:07.226121 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.226002    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec6b7d20-2ce7-4819-82bf-36a12f08275b-tmp\") pod \"klusterlet-addon-workmgr-98b95c4fd-srl79\" (UID: \"ec6b7d20-2ce7-4819-82bf-36a12f08275b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:14:07.226432 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.226410    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec6b7d20-2ce7-4819-82bf-36a12f08275b-tmp\") pod \"klusterlet-addon-workmgr-98b95c4fd-srl79\" (UID: \"ec6b7d20-2ce7-4819-82bf-36a12f08275b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:14:07.230262 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.230232    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ec6b7d20-2ce7-4819-82bf-36a12f08275b-klusterlet-config\") pod \"klusterlet-addon-workmgr-98b95c4fd-srl79\" (UID: \"ec6b7d20-2ce7-4819-82bf-36a12f08275b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:14:07.234482 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.234459    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5c78\" (UniqueName: \"kubernetes.io/projected/ec6b7d20-2ce7-4819-82bf-36a12f08275b-kube-api-access-q5c78\") pod \"klusterlet-addon-workmgr-98b95c4fd-srl79\" (UID: \"ec6b7d20-2ce7-4819-82bf-36a12f08275b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:14:07.326903 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.326803    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:14:07.327087 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:07.326938    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 18:14:07.327087 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.326941    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:14:07.327087 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:07.326998    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls podName:d223e041-067c-49a8-a19c-b4879963bc0e nodeName:}" failed. No retries permitted until 2026-07-20 18:14:15.326980098 +0000 UTC m=+52.160952470 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls") pod "dns-default-85nj6" (UID: "d223e041-067c-49a8-a19c-b4879963bc0e") : secret "dns-default-metrics-tls" not found
Jul 20 18:14:07.327087 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:07.327050    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 18:14:07.327087 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:07.327080    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5875d48c7b-kl9md: secret "image-registry-tls" not found
Jul 20 18:14:07.327354 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:07.327137    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls podName:6fe2898d-c51c-4fe4-a2c5-c554e9e499be nodeName:}" failed. No retries permitted until 2026-07-20 18:14:15.327123972 +0000 UTC m=+52.161096338 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls") pod "image-registry-5875d48c7b-kl9md" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be") : secret "image-registry-tls" not found
Jul 20 18:14:07.332771 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.332751    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:14:07.470951 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.470915    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"]
Jul 20 18:14:07.474426 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:14:07.474399    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec6b7d20_2ce7_4819_82bf_36a12f08275b.slice/crio-06d13096a3d18441d4d49665a03216fd8d5101d4483ea06d35b630300bde0b85 WatchSource:0}: Error finding container 06d13096a3d18441d4d49665a03216fd8d5101d4483ea06d35b630300bde0b85: Status 404 returned error can't find the container with id 06d13096a3d18441d4d49665a03216fd8d5101d4483ea06d35b630300bde0b85
Jul 20 18:14:07.704304 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.704230    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:07.806839 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.806807    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:07.907711 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.907680    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:07.965915 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:07.965840    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:08.028600 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:08.028562    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79" event={"ID":"ec6b7d20-2ce7-4819-82bf-36a12f08275b","Type":"ContainerStarted","Data":"06d13096a3d18441d4d49665a03216fd8d5101d4483ea06d35b630300bde0b85"}
Jul 20 18:14:08.031760 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:08.031735    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq7xn" event={"ID":"f2c72dbe-b803-492b-9046-eeeacf69b9cf","Type":"ContainerStarted","Data":"b9a7e88af2e4679bb7d3c0fec2f7d5030c7fd63352e8edf77751c73179c05293"}
Jul 20 18:14:08.055034 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:08.054984    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pq7xn" podStartSLOduration=13.366939851 podStartE2EDuration="44.054968649s" totalImagesPullingTime="30.688028798s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 18:13:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:14:08.054916827 +0000 UTC m=+44.888889228" watchObservedRunningTime="2026-07-20 18:14:08.054968649 +0000 UTC m=+44.888941036"
Jul 20 18:14:08.675684 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:08.675654    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:08.744676 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:08.744643    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:09.080752 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:09.080721    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:09.134836 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:09.134796    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:10.819120 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:10.818914    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:10.920549 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:10.920510    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:12.737664 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:12.737627    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:12.798367 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:12.798335    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:15.390297 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:15.390254    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:14:15.390708 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:15.390347    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:14:15.390708 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:15.390435    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 18:14:15.390708 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:15.390454    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5875d48c7b-kl9md: secret "image-registry-tls" not found
Jul 20 18:14:15.390708 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:15.390473    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 18:14:15.390708 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:15.390530    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls podName:6fe2898d-c51c-4fe4-a2c5-c554e9e499be nodeName:}" failed. No retries permitted until 2026-07-20 18:14:31.390507276 +0000 UTC m=+68.224479670 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls") pod "image-registry-5875d48c7b-kl9md" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be") : secret "image-registry-tls" not found
Jul 20 18:14:15.390708 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:15.390546    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls podName:d223e041-067c-49a8-a19c-b4879963bc0e nodeName:}" failed. No retries permitted until 2026-07-20 18:14:31.390539659 +0000 UTC m=+68.224512025 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls") pod "dns-default-85nj6" (UID: "d223e041-067c-49a8-a19c-b4879963bc0e") : secret "dns-default-metrics-tls" not found
Jul 20 18:14:18.057441 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:18.057404    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79" event={"ID":"ec6b7d20-2ce7-4819-82bf-36a12f08275b","Type":"ContainerStarted","Data":"e44a140409a72ec407f6e757a156180ea1a44e55dee709fabe3ca1940ddb6a8a"}
Jul 20 18:14:18.057933 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:18.057645    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:14:18.059272 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:18.059251    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:14:18.074010 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:18.073965    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79" podStartSLOduration=2.32614146 podStartE2EDuration="12.073947192s" totalImagesPullingTime="9.747805732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:14:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:14:18.073328825 +0000 UTC m=+54.907301226" watchObservedRunningTime="2026-07-20 18:14:18.073947192 +0000 UTC m=+54.907919583"
Jul 20 18:14:23.816729 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:23.816682    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:23.871911 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:23.871874    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:25.996655 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:25.996629    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8f6m2"
Jul 20 18:14:26.153555 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:26.153520    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:26.254347 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:26.254250    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:28.683740 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:28.683692    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:28.743002 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:28.742967    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:29.389223 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:29.389185    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret\") pod \"global-pull-secret-syncer-pjhww\" (UID: \"31d6832c-6d75-41e8-b272-7edeb41be490\") " pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:14:29.392363 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:29.392345    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 18:14:29.402322 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:29.402292    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31d6832c-6d75-41e8-b272-7edeb41be490-original-pull-secret\") pod \"global-pull-secret-syncer-pjhww\" (UID: \"31d6832c-6d75-41e8-b272-7edeb41be490\") " pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:14:29.464389 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:29.464348    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjhww"
Jul 20 18:14:29.490416 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:29.490383    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs\") pod \"network-metrics-daemon-g4z7w\" (UID: \"1bae9e54-a7d1-46ba-9bb6-9602c8d53962\") " pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:14:29.492751 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:29.492731    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 18:14:29.500842 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:29.500722    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 20 18:14:29.500842 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:29.500799    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs podName:1bae9e54-a7d1-46ba-9bb6-9602c8d53962 nodeName:}" failed. No retries permitted until 2026-07-20 18:15:33.500776399 +0000 UTC m=+130.334748779 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs") pod "network-metrics-daemon-g4z7w" (UID: "1bae9e54-a7d1-46ba-9bb6-9602c8d53962") : secret "metrics-daemon-secret" not found
Jul 20 18:14:29.590933 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:29.590903    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmtrm\" (UniqueName: \"kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm\") pod \"network-check-target-hb8sv\" (UID: \"dd6d9efb-9a7b-4247-95c0-e8a3a56600eb\") " pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:14:29.593774 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:29.593750    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 18:14:29.594313 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:29.594293    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pjhww"]
Jul 20 18:14:29.597072 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:14:29.597034    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31d6832c_6d75_41e8_b272_7edeb41be490.slice/crio-678c1bbdb0924766e4cf4eab606d446734b7a68b307587b446bc935667515a9e WatchSource:0}: Error finding container 678c1bbdb0924766e4cf4eab606d446734b7a68b307587b446bc935667515a9e: Status 404 returned error can't find the container with id 678c1bbdb0924766e4cf4eab606d446734b7a68b307587b446bc935667515a9e
Jul 20 18:14:29.603373 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:29.603356    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 18:14:29.614602 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:29.614583    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmtrm\" (UniqueName: \"kubernetes.io/projected/dd6d9efb-9a7b-4247-95c0-e8a3a56600eb-kube-api-access-rmtrm\") pod \"network-check-target-hb8sv\" (UID: \"dd6d9efb-9a7b-4247-95c0-e8a3a56600eb\") " pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:14:29.795014 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:29.794984    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ls8qp\""
Jul 20 18:14:29.802940 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:29.802923    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:14:29.920749 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:29.920720    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hb8sv"]
Jul 20 18:14:29.924753 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:14:29.924728    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd6d9efb_9a7b_4247_95c0_e8a3a56600eb.slice/crio-0bd335f042bcc6165bd5d997f5fd7014261f30e52f4bf659825946967de66fb9 WatchSource:0}: Error finding container 0bd335f042bcc6165bd5d997f5fd7014261f30e52f4bf659825946967de66fb9: Status 404 returned error can't find the container with id 0bd335f042bcc6165bd5d997f5fd7014261f30e52f4bf659825946967de66fb9
Jul 20 18:14:30.086413 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:30.086332    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hb8sv" event={"ID":"dd6d9efb-9a7b-4247-95c0-e8a3a56600eb","Type":"ContainerStarted","Data":"0bd335f042bcc6165bd5d997f5fd7014261f30e52f4bf659825946967de66fb9"}
Jul 20 18:14:30.087384 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:30.087358    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pjhww" event={"ID":"31d6832c-6d75-41e8-b272-7edeb41be490","Type":"ContainerStarted","Data":"678c1bbdb0924766e4cf4eab606d446734b7a68b307587b446bc935667515a9e"}
Jul 20 18:14:31.404172 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:31.404131    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:14:31.404590 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:31.404226    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:14:31.404590 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:31.404323    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 18:14:31.404590 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:31.404347    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5875d48c7b-kl9md: secret "image-registry-tls" not found
Jul 20 18:14:31.404590 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:31.404349    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 18:14:31.404590 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:31.404406    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls podName:d223e041-067c-49a8-a19c-b4879963bc0e nodeName:}" failed. No retries permitted until 2026-07-20 18:15:03.404387508 +0000 UTC m=+100.238359877 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls") pod "dns-default-85nj6" (UID: "d223e041-067c-49a8-a19c-b4879963bc0e") : secret "dns-default-metrics-tls" not found
Jul 20 18:14:31.404590 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:14:31.404422    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls podName:6fe2898d-c51c-4fe4-a2c5-c554e9e499be nodeName:}" failed. No retries permitted until 2026-07-20 18:15:03.404413385 +0000 UTC m=+100.238385754 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls") pod "image-registry-5875d48c7b-kl9md" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be") : secret "image-registry-tls" not found
Jul 20 18:14:35.103420 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:35.103384    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pjhww" event={"ID":"31d6832c-6d75-41e8-b272-7edeb41be490","Type":"ContainerStarted","Data":"5ee3a96e3b712d7e613aa3ee7442c2252d754854c27438efcbea2eef9bb7e038"}
Jul 20 18:14:35.104857 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:35.104827    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hb8sv" event={"ID":"dd6d9efb-9a7b-4247-95c0-e8a3a56600eb","Type":"ContainerStarted","Data":"75b3c2fd4b728d2051cac03406b158149790db20054fa68909a996c81c232819"}
Jul 20 18:14:35.104967 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:35.104956    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:14:35.119093 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:35.119035    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pjhww" podStartSLOduration=66.197959612 podStartE2EDuration="1m11.119021889s" totalImagesPullingTime="4.921062277s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 18:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:14:35.118497723 +0000 UTC m=+71.952470112" watchObservedRunningTime="2026-07-20 18:14:35.119021889 +0000 UTC m=+71.952994276"
Jul 20 18:14:35.136743 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:35.136707    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hb8sv" podStartSLOduration=66.550653823 podStartE2EDuration="1m11.136696537s" totalImagesPullingTime="4.586042714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:14:35.136257741 +0000 UTC m=+71.970230129" watchObservedRunningTime="2026-07-20 18:14:35.136696537 +0000 UTC m=+71.970668954"
Jul 20 18:14:39.725642 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:39.725609    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:14:39.826694 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:14:39.826646    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:15:03.438252 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:15:03.438214    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:15:03.438640 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:15:03.438271    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:15:03.438640 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:15:03.438357    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 18:15:03.438640 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:15:03.438360    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 18:15:03.438640 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:15:03.438378    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5875d48c7b-kl9md: secret "image-registry-tls" not found
Jul 20 18:15:03.438640 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:15:03.438415    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls podName:d223e041-067c-49a8-a19c-b4879963bc0e nodeName:}" failed. No retries permitted until 2026-07-20 18:16:07.438401898 +0000 UTC m=+164.272374264 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls") pod "dns-default-85nj6" (UID: "d223e041-067c-49a8-a19c-b4879963bc0e") : secret "dns-default-metrics-tls" not found
Jul 20 18:15:03.438640 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:15:03.438428    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls podName:6fe2898d-c51c-4fe4-a2c5-c554e9e499be nodeName:}" failed. No retries permitted until 2026-07-20 18:16:07.43842191 +0000 UTC m=+164.272394275 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls") pod "image-registry-5875d48c7b-kl9md" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be") : secret "image-registry-tls" not found
Jul 20 18:15:04.079277 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:15:04.079191    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:15:06.108904 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:15:06.108876    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hb8sv"
Jul 20 18:15:21.564699 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:15:21.564663    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:15:33.552563 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:15:33.552522    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs\") pod \"network-metrics-daemon-g4z7w\" (UID: \"1bae9e54-a7d1-46ba-9bb6-9602c8d53962\") " pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:15:33.552985 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:15:33.552636    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 20 18:15:33.552985 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:15:33.552689    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs podName:1bae9e54-a7d1-46ba-9bb6-9602c8d53962 nodeName:}" failed. No retries permitted until 2026-07-20 18:17:35.552675486 +0000 UTC m=+252.386647853 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs") pod "network-metrics-daemon-g4z7w" (UID: "1bae9e54-a7d1-46ba-9bb6-9602c8d53962") : secret "metrics-daemon-secret" not found
Jul 20 18:16:02.570713 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:16:02.570668    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md" podUID="6fe2898d-c51c-4fe4-a2c5-c554e9e499be"
Jul 20 18:16:02.583864 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:16:02.583830    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-85nj6" podUID="d223e041-067c-49a8-a19c-b4879963bc0e"
Jul 20 18:16:02.773070 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:16:02.773011    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:16:03.310898 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:16:03.310869    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-85nj6"
Jul 20 18:16:03.311080 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:16:03.310869    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:16:07.496680 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:16:07.496638    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:16:07.497148 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:16:07.496693    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:16:07.497148 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:16:07.496795    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 18:16:07.497148 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:16:07.496805    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5875d48c7b-kl9md: secret "image-registry-tls" not found
Jul 20 18:16:07.497148 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:16:07.496819    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 18:16:07.497148 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:16:07.496866    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls podName:6fe2898d-c51c-4fe4-a2c5-c554e9e499be nodeName:}" failed. No retries permitted until 2026-07-20 18:18:09.496851575 +0000 UTC m=+286.330823941 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls") pod "image-registry-5875d48c7b-kl9md" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be") : secret "image-registry-tls" not found
Jul 20 18:16:07.497148 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:16:07.496894    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls podName:d223e041-067c-49a8-a19c-b4879963bc0e nodeName:}" failed. No retries permitted until 2026-07-20 18:18:09.49687176 +0000 UTC m=+286.330844127 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls") pod "dns-default-85nj6" (UID: "d223e041-067c-49a8-a19c-b4879963bc0e") : secret "dns-default-metrics-tls" not found
Jul 20 18:16:17.333322 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:16:17.333281    2569 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79" podUID="ec6b7d20-2ce7-4819-82bf-36a12f08275b" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.9:8000/healthz\": dial tcp 10.133.0.9:8000: connect: connection refused"
Jul 20 18:16:17.344423 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:16:17.344395    2569 generic.go:356] "Generic (PLEG): container finished" podID="ec6b7d20-2ce7-4819-82bf-36a12f08275b" containerID="e44a140409a72ec407f6e757a156180ea1a44e55dee709fabe3ca1940ddb6a8a" exitCode=1
Jul 20 18:16:17.344527 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:16:17.344433    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79" event={"ID":"ec6b7d20-2ce7-4819-82bf-36a12f08275b","Type":"ContainerDied","Data":"e44a140409a72ec407f6e757a156180ea1a44e55dee709fabe3ca1940ddb6a8a"}
Jul 20 18:16:17.344731 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:16:17.344718    2569 scope.go:117] "RemoveContainer" containerID="e44a140409a72ec407f6e757a156180ea1a44e55dee709fabe3ca1940ddb6a8a"
Jul 20 18:16:17.753106 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:16:17.753082    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:16:18.058176 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:16:18.058085    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:16:18.348267 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:16:18.348178    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79" event={"ID":"ec6b7d20-2ce7-4819-82bf-36a12f08275b","Type":"ContainerStarted","Data":"7fe67f36d0cc75c45532e0c8cb8d62cd045a811dd005723f22078c4c227300bd"}
Jul 20 18:16:18.348698 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:16:18.348391    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:16:18.349297 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:16:18.349277    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:17:19.508806 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.508774    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-t7pmk"]
Jul 20 18:17:19.510764 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.510748    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.512907 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.512880    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 20 18:17:19.513077 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.512971    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 20 18:17:19.513077 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.513016    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 20 18:17:19.513339 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.513306    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 20 18:17:19.513891 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.513879    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-t5fjr\""
Jul 20 18:17:19.521380 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.521362    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 20 18:17:19.559521 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.559482    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-t7pmk"]
Jul 20 18:17:19.600111 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.600075    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd"]
Jul 20 18:17:19.602301 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.602280    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-pb2nw"]
Jul 20 18:17:19.602417 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.602389    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162cbe48-6490-47b2-b881-85d8e8e95e19-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.602491 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.602419    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd"
Jul 20 18:17:19.602491 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.602431    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/162cbe48-6490-47b2-b881-85d8e8e95e19-tmp\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.602491 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.602456    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162cbe48-6490-47b2-b881-85d8e8e95e19-service-ca-bundle\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.602642 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.602559    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/162cbe48-6490-47b2-b881-85d8e8e95e19-serving-cert\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.602642 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.602617    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/162cbe48-6490-47b2-b881-85d8e8e95e19-snapshots\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.602798 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.602647    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fhj7\" (UniqueName: \"kubernetes.io/projected/162cbe48-6490-47b2-b881-85d8e8e95e19-kube-api-access-9fhj7\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.604227 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.604209    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:19.606626 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.606589    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 20 18:17:19.606626 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.606624    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 20 18:17:19.606882 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.606596    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 20 18:17:19.606882 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.606760    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 20 18:17:19.606882 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.606841    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 20 18:17:19.607110 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.606920    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-f8f9j\""
Jul 20 18:17:19.607110 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.606942    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 20 18:17:19.607230 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.607215    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 20 18:17:19.607424 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.607407    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-7gwkh\""
Jul 20 18:17:19.607523 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.607408    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 20 18:17:19.613157 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.613100    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 20 18:17:19.614738 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.614717    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd"]
Jul 20 18:17:19.618567 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.618547    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-pb2nw"]
Jul 20 18:17:19.703460 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.703428    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162cbe48-6490-47b2-b881-85d8e8e95e19-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.703460 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.703469    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/162cbe48-6490-47b2-b881-85d8e8e95e19-tmp\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.703704 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.703497    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfzqj\" (UniqueName: \"kubernetes.io/projected/c4f56323-217f-4fe1-be14-ec9ad4b7b81a-kube-api-access-bfzqj\") pod \"console-operator-565b7b569d-pb2nw\" (UID: \"c4f56323-217f-4fe1-be14-ec9ad4b7b81a\") " pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:19.703704 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.703531    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/162cbe48-6490-47b2-b881-85d8e8e95e19-serving-cert\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.703704 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.703553    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c4f56323-217f-4fe1-be14-ec9ad4b7b81a-serving-cert\") pod \"console-operator-565b7b569d-pb2nw\" (UID: \"c4f56323-217f-4fe1-be14-ec9ad4b7b81a\") " pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:19.703704 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.703582    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c1d457d-6952-4712-a0f3-a4b769db92fa-serving-cert\") pod \"service-ca-operator-5bcbc94c78-qpzjd\" (UID: \"0c1d457d-6952-4712-a0f3-a4b769db92fa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd"
Jul 20 18:17:19.703704 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.703670    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fhj7\" (UniqueName: \"kubernetes.io/projected/162cbe48-6490-47b2-b881-85d8e8e95e19-kube-api-access-9fhj7\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.703933 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.703722    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4ss4\" (UniqueName: \"kubernetes.io/projected/0c1d457d-6952-4712-a0f3-a4b769db92fa-kube-api-access-g4ss4\") pod \"service-ca-operator-5bcbc94c78-qpzjd\" (UID: \"0c1d457d-6952-4712-a0f3-a4b769db92fa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd"
Jul 20 18:17:19.703933 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.703769    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c4f56323-217f-4fe1-be14-ec9ad4b7b81a-trusted-ca\") pod \"console-operator-565b7b569d-pb2nw\" (UID: \"c4f56323-217f-4fe1-be14-ec9ad4b7b81a\") " pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:19.703933 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.703814    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162cbe48-6490-47b2-b881-85d8e8e95e19-service-ca-bundle\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.703933 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.703896    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/162cbe48-6490-47b2-b881-85d8e8e95e19-tmp\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.703933 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.703919    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/162cbe48-6490-47b2-b881-85d8e8e95e19-snapshots\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.704156 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.703961    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c1d457d-6952-4712-a0f3-a4b769db92fa-config\") pod \"service-ca-operator-5bcbc94c78-qpzjd\" (UID: \"0c1d457d-6952-4712-a0f3-a4b769db92fa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd"
Jul 20 18:17:19.704156 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.703986    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c4f56323-217f-4fe1-be14-ec9ad4b7b81a-config\") pod \"console-operator-565b7b569d-pb2nw\" (UID: \"c4f56323-217f-4fe1-be14-ec9ad4b7b81a\") " pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:19.704317 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.704298    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162cbe48-6490-47b2-b881-85d8e8e95e19-service-ca-bundle\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.704317 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.704310    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162cbe48-6490-47b2-b881-85d8e8e95e19-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.704466 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.704447    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/162cbe48-6490-47b2-b881-85d8e8e95e19-snapshots\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.705842 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.705826    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/162cbe48-6490-47b2-b881-85d8e8e95e19-serving-cert\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.718691 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.718657    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fhj7\" (UniqueName: \"kubernetes.io/projected/162cbe48-6490-47b2-b881-85d8e8e95e19-kube-api-access-9fhj7\") pod \"insights-operator-555d9d7885-t7pmk\" (UID: \"162cbe48-6490-47b2-b881-85d8e8e95e19\") " pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.782392 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.782314    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:19.804300 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.804270    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c1d457d-6952-4712-a0f3-a4b769db92fa-serving-cert\") pod \"service-ca-operator-5bcbc94c78-qpzjd\" (UID: \"0c1d457d-6952-4712-a0f3-a4b769db92fa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd"
Jul 20 18:17:19.804434 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.804311    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4ss4\" (UniqueName: \"kubernetes.io/projected/0c1d457d-6952-4712-a0f3-a4b769db92fa-kube-api-access-g4ss4\") pod \"service-ca-operator-5bcbc94c78-qpzjd\" (UID: \"0c1d457d-6952-4712-a0f3-a4b769db92fa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd"
Jul 20 18:17:19.804434 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.804352    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c4f56323-217f-4fe1-be14-ec9ad4b7b81a-trusted-ca\") pod \"console-operator-565b7b569d-pb2nw\" (UID: \"c4f56323-217f-4fe1-be14-ec9ad4b7b81a\") " pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:19.804434 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.804413    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c1d457d-6952-4712-a0f3-a4b769db92fa-config\") pod \"service-ca-operator-5bcbc94c78-qpzjd\" (UID: \"0c1d457d-6952-4712-a0f3-a4b769db92fa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd"
Jul 20 18:17:19.804635 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.804434    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c4f56323-217f-4fe1-be14-ec9ad4b7b81a-config\") pod \"console-operator-565b7b569d-pb2nw\" (UID: \"c4f56323-217f-4fe1-be14-ec9ad4b7b81a\") " pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:19.804635 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.804467    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfzqj\" (UniqueName: \"kubernetes.io/projected/c4f56323-217f-4fe1-be14-ec9ad4b7b81a-kube-api-access-bfzqj\") pod \"console-operator-565b7b569d-pb2nw\" (UID: \"c4f56323-217f-4fe1-be14-ec9ad4b7b81a\") " pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:19.804635 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.804499    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c4f56323-217f-4fe1-be14-ec9ad4b7b81a-serving-cert\") pod \"console-operator-565b7b569d-pb2nw\" (UID: \"c4f56323-217f-4fe1-be14-ec9ad4b7b81a\") " pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:19.805109 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.805078    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c1d457d-6952-4712-a0f3-a4b769db92fa-config\") pod \"service-ca-operator-5bcbc94c78-qpzjd\" (UID: \"0c1d457d-6952-4712-a0f3-a4b769db92fa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd"
Jul 20 18:17:19.805276 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.805253    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c4f56323-217f-4fe1-be14-ec9ad4b7b81a-config\") pod \"console-operator-565b7b569d-pb2nw\" (UID: \"c4f56323-217f-4fe1-be14-ec9ad4b7b81a\") " pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:19.805357 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.805291    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c4f56323-217f-4fe1-be14-ec9ad4b7b81a-trusted-ca\") pod \"console-operator-565b7b569d-pb2nw\" (UID: \"c4f56323-217f-4fe1-be14-ec9ad4b7b81a\") " pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:19.806711 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.806689    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c1d457d-6952-4712-a0f3-a4b769db92fa-serving-cert\") pod \"service-ca-operator-5bcbc94c78-qpzjd\" (UID: \"0c1d457d-6952-4712-a0f3-a4b769db92fa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd"
Jul 20 18:17:19.806804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.806772    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c4f56323-217f-4fe1-be14-ec9ad4b7b81a-serving-cert\") pod \"console-operator-565b7b569d-pb2nw\" (UID: \"c4f56323-217f-4fe1-be14-ec9ad4b7b81a\") " pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:19.811689 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.811673    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4ss4\" (UniqueName: \"kubernetes.io/projected/0c1d457d-6952-4712-a0f3-a4b769db92fa-kube-api-access-g4ss4\") pod \"service-ca-operator-5bcbc94c78-qpzjd\" (UID: \"0c1d457d-6952-4712-a0f3-a4b769db92fa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd"
Jul 20 18:17:19.812204 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.812187    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfzqj\" (UniqueName: \"kubernetes.io/projected/c4f56323-217f-4fe1-be14-ec9ad4b7b81a-kube-api-access-bfzqj\") pod \"console-operator-565b7b569d-pb2nw\" (UID: \"c4f56323-217f-4fe1-be14-ec9ad4b7b81a\") " pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:19.819987 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.819968    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-t7pmk"
Jul 20 18:17:19.871398 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.871349    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:19.912562 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.912529    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd"
Jul 20 18:17:19.918621 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.918594    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:19.939659 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:19.939633    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-t7pmk"]
Jul 20 18:17:19.945127 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:17:19.945090    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod162cbe48_6490_47b2_b881_85d8e8e95e19.slice/crio-d3c932d5a9ea740a90ca8e0b1c21a7f4caae9e00a6d6805942c66c4b01097ac2 WatchSource:0}: Error finding container d3c932d5a9ea740a90ca8e0b1c21a7f4caae9e00a6d6805942c66c4b01097ac2: Status 404 returned error can't find the container with id d3c932d5a9ea740a90ca8e0b1c21a7f4caae9e00a6d6805942c66c4b01097ac2
Jul 20 18:17:20.040734 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:20.040662    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd"]
Jul 20 18:17:20.043768 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:17:20.043741    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c1d457d_6952_4712_a0f3_a4b769db92fa.slice/crio-001a71b3ee537dd2552cc5fb59cdacf53697ac1030ab716e45e186e56845865a WatchSource:0}: Error finding container 001a71b3ee537dd2552cc5fb59cdacf53697ac1030ab716e45e186e56845865a: Status 404 returned error can't find the container with id 001a71b3ee537dd2552cc5fb59cdacf53697ac1030ab716e45e186e56845865a
Jul 20 18:17:20.055444 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:20.055425    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-pb2nw"]
Jul 20 18:17:20.057851 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:17:20.057830    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4f56323_217f_4fe1_be14_ec9ad4b7b81a.slice/crio-f6fa86c404e21586164e45a0ece577c3c8510b4013485542885af401e0d85601 WatchSource:0}: Error finding container f6fa86c404e21586164e45a0ece577c3c8510b4013485542885af401e0d85601: Status 404 returned error can't find the container with id f6fa86c404e21586164e45a0ece577c3c8510b4013485542885af401e0d85601
Jul 20 18:17:20.496650 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:20.496613    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw" event={"ID":"c4f56323-217f-4fe1-be14-ec9ad4b7b81a","Type":"ContainerStarted","Data":"f6fa86c404e21586164e45a0ece577c3c8510b4013485542885af401e0d85601"}
Jul 20 18:17:20.497683 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:20.497658    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd" event={"ID":"0c1d457d-6952-4712-a0f3-a4b769db92fa","Type":"ContainerStarted","Data":"001a71b3ee537dd2552cc5fb59cdacf53697ac1030ab716e45e186e56845865a"}
Jul 20 18:17:20.498594 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:20.498575    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-t7pmk" event={"ID":"162cbe48-6490-47b2-b881-85d8e8e95e19","Type":"ContainerStarted","Data":"d3c932d5a9ea740a90ca8e0b1c21a7f4caae9e00a6d6805942c66c4b01097ac2"}
Jul 20 18:17:22.267727 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:22.267686    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:23.488822 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.488791    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:23.492156 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.492127    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-w9t2q"]
Jul 20 18:17:23.495086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.495069    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-w9t2q"
Jul 20 18:17:23.501002 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.500843    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 20 18:17:23.501812 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.501791    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-gqc46\""
Jul 20 18:17:23.506310 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.506281    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 20 18:17:23.508404 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.508380    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd" event={"ID":"0c1d457d-6952-4712-a0f3-a4b769db92fa","Type":"ContainerStarted","Data":"b53e3264fd528ecdbde31548b66e438be4cd5d46315a591488fb244d93bbf03a"}
Jul 20 18:17:23.509970 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.509946    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-t7pmk" event={"ID":"162cbe48-6490-47b2-b881-85d8e8e95e19","Type":"ContainerStarted","Data":"2afe71d3318016d7c9e5125dc9a43cd4f81b44713b5671fc64ebd6325a7f1de9"}
Jul 20 18:17:23.511395 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.511377    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/0.log"
Jul 20 18:17:23.511490 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.511412    2569 generic.go:356] "Generic (PLEG): container finished" podID="c4f56323-217f-4fe1-be14-ec9ad4b7b81a" containerID="8bca1167ea44d08bc972eaa302a7cbdc86b654b7f0c6c83792ef387ec76a3f65" exitCode=255
Jul 20 18:17:23.511490 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.511454    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw" event={"ID":"c4f56323-217f-4fe1-be14-ec9ad4b7b81a","Type":"ContainerDied","Data":"8bca1167ea44d08bc972eaa302a7cbdc86b654b7f0c6c83792ef387ec76a3f65"}
Jul 20 18:17:23.511626 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.511613    2569 scope.go:117] "RemoveContainer" containerID="8bca1167ea44d08bc972eaa302a7cbdc86b654b7f0c6c83792ef387ec76a3f65"
Jul 20 18:17:23.514651 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.514629    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-w9t2q"]
Jul 20 18:17:23.535420 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.535393    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlqzw\" (UniqueName: \"kubernetes.io/projected/f43ff13a-ead5-45d1-9e43-b17ddef43cf5-kube-api-access-rlqzw\") pod \"migrator-7cc9c7ffd7-w9t2q\" (UID: \"f43ff13a-ead5-45d1-9e43-b17ddef43cf5\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-w9t2q"
Jul 20 18:17:23.581614 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.581554    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-t7pmk" podStartSLOduration=1.860280028 podStartE2EDuration="4.5815359s" totalImagesPullingTime="2.721255872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:17:23.580434983 +0000 UTC m=+240.414407371" watchObservedRunningTime="2026-07-20 18:17:23.5815359 +0000 UTC m=+240.415508289"
Jul 20 18:17:23.597203 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.597089    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd" podStartSLOduration=1.300040216 podStartE2EDuration="4.597068186s" totalImagesPullingTime="3.29702797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:17:23.596990642 +0000 UTC m=+240.430963032" watchObservedRunningTime="2026-07-20 18:17:23.597068186 +0000 UTC m=+240.431040573"
Jul 20 18:17:23.600161 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.600136    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:23.638464 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.637114    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlqzw\" (UniqueName: \"kubernetes.io/projected/f43ff13a-ead5-45d1-9e43-b17ddef43cf5-kube-api-access-rlqzw\") pod \"migrator-7cc9c7ffd7-w9t2q\" (UID: \"f43ff13a-ead5-45d1-9e43-b17ddef43cf5\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-w9t2q"
Jul 20 18:17:23.647598 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.647574    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 20 18:17:23.658422 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.658403    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 20 18:17:23.668115 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.668092    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlqzw\" (UniqueName: \"kubernetes.io/projected/f43ff13a-ead5-45d1-9e43-b17ddef43cf5-kube-api-access-rlqzw\") pod \"migrator-7cc9c7ffd7-w9t2q\" (UID: \"f43ff13a-ead5-45d1-9e43-b17ddef43cf5\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-w9t2q"
Jul 20 18:17:23.697148 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.697115    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:23.806184 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.806150    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-gqc46\""
Jul 20 18:17:23.814363 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.814325    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-w9t2q"
Jul 20 18:17:23.850831 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.850750    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7q9t5"]
Jul 20 18:17:23.853695 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.853673    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:23.856011 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.855981    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 20 18:17:23.856132 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.855985    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-928kt\""
Jul 20 18:17:23.856333 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.856315    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 20 18:17:23.869395 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.869366    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7q9t5"]
Jul 20 18:17:23.886710 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.886318    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:23.939815 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.939783    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/629056e9-18ee-4294-a8ab-42081cff41b4-data-volume\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:23.939991 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.939934    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/629056e9-18ee-4294-a8ab-42081cff41b4-crio-socket\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:23.939991 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.939980    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:23.940102 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.940018    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:23.940102 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.940079    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf5bp\" (UniqueName: \"kubernetes.io/projected/629056e9-18ee-4294-a8ab-42081cff41b4-kube-api-access-xf5bp\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:23.961153 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:23.961110    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-w9t2q"]
Jul 20 18:17:23.965157 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:17:23.965131    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf43ff13a_ead5_45d1_9e43_b17ddef43cf5.slice/crio-eff06a2e3f0c00a97ce3c29a44a128b1f933b4496c79badc0bce8a7db20fb885 WatchSource:0}: Error finding container eff06a2e3f0c00a97ce3c29a44a128b1f933b4496c79badc0bce8a7db20fb885: Status 404 returned error can't find the container with id eff06a2e3f0c00a97ce3c29a44a128b1f933b4496c79badc0bce8a7db20fb885
Jul 20 18:17:24.040926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.040896    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:24.041157 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.040934    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf5bp\" (UniqueName: \"kubernetes.io/projected/629056e9-18ee-4294-a8ab-42081cff41b4-kube-api-access-xf5bp\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:24.041157 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.040957    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/629056e9-18ee-4294-a8ab-42081cff41b4-data-volume\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:24.041157 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:24.041088    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 18:17:24.041157 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:24.041159    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls podName:629056e9-18ee-4294-a8ab-42081cff41b4 nodeName:}" failed. No retries permitted until 2026-07-20 18:17:24.541139364 +0000 UTC m=+241.375111747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7q9t5" (UID: "629056e9-18ee-4294-a8ab-42081cff41b4") : secret "insights-runtime-extractor-tls" not found
Jul 20 18:17:24.041392 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.041214    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/629056e9-18ee-4294-a8ab-42081cff41b4-crio-socket\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:24.041392 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.041255    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:24.041392 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.041327    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/629056e9-18ee-4294-a8ab-42081cff41b4-crio-socket\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:24.041392 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:24.041341    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 18:17:24.041392 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.041330    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/629056e9-18ee-4294-a8ab-42081cff41b4-data-volume\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:24.041392 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:24.041387    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm podName:629056e9-18ee-4294-a8ab-42081cff41b4 nodeName:}" failed. No retries permitted until 2026-07-20 18:17:24.541373764 +0000 UTC m=+241.375346134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7q9t5" (UID: "629056e9-18ee-4294-a8ab-42081cff41b4") : configmap "kube-rbac-proxy" not found
Jul 20 18:17:24.055983 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.055956    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf5bp\" (UniqueName: \"kubernetes.io/projected/629056e9-18ee-4294-a8ab-42081cff41b4-kube-api-access-xf5bp\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:24.511229 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.511195    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:24.515914 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.515893    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/1.log"
Jul 20 18:17:24.516303 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.516288    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/0.log"
Jul 20 18:17:24.516358 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.516325    2569 generic.go:356] "Generic (PLEG): container finished" podID="c4f56323-217f-4fe1-be14-ec9ad4b7b81a" containerID="68c0799749f94a8763b08f6846984fab89f363e21501b4c49f1ae432cce82583" exitCode=255
Jul 20 18:17:24.516450 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.516422    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw" event={"ID":"c4f56323-217f-4fe1-be14-ec9ad4b7b81a","Type":"ContainerDied","Data":"68c0799749f94a8763b08f6846984fab89f363e21501b4c49f1ae432cce82583"}
Jul 20 18:17:24.516503 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.516489    2569 scope.go:117] "RemoveContainer" containerID="8bca1167ea44d08bc972eaa302a7cbdc86b654b7f0c6c83792ef387ec76a3f65"
Jul 20 18:17:24.516736 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.516721    2569 scope.go:117] "RemoveContainer" containerID="68c0799749f94a8763b08f6846984fab89f363e21501b4c49f1ae432cce82583"
Jul 20 18:17:24.516932 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:24.516911    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-pb2nw_openshift-console-operator(c4f56323-217f-4fe1-be14-ec9ad4b7b81a)\"" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw" podUID="c4f56323-217f-4fe1-be14-ec9ad4b7b81a"
Jul 20 18:17:24.517492 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.517449    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-w9t2q" event={"ID":"f43ff13a-ead5-45d1-9e43-b17ddef43cf5","Type":"ContainerStarted","Data":"eff06a2e3f0c00a97ce3c29a44a128b1f933b4496c79badc0bce8a7db20fb885"}
Jul 20 18:17:24.545783 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.545752    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:24.545930 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:24.545797    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:24.545971 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:24.545922    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 18:17:24.546012 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:24.546001    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls podName:629056e9-18ee-4294-a8ab-42081cff41b4 nodeName:}" failed. No retries permitted until 2026-07-20 18:17:25.545973004 +0000 UTC m=+242.379945370 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7q9t5" (UID: "629056e9-18ee-4294-a8ab-42081cff41b4") : secret "insights-runtime-extractor-tls" not found
Jul 20 18:17:24.546079 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:24.546027    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 18:17:24.546119 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:24.546097    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm podName:629056e9-18ee-4294-a8ab-42081cff41b4 nodeName:}" failed. No retries permitted until 2026-07-20 18:17:25.546081423 +0000 UTC m=+242.380053790 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7q9t5" (UID: "629056e9-18ee-4294-a8ab-42081cff41b4") : configmap "kube-rbac-proxy" not found
Jul 20 18:17:25.523028 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:25.523004    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/1.log"
Jul 20 18:17:25.523491 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:25.523350    2569 scope.go:117] "RemoveContainer" containerID="68c0799749f94a8763b08f6846984fab89f363e21501b4c49f1ae432cce82583"
Jul 20 18:17:25.523539 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:25.523521    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-pb2nw_openshift-console-operator(c4f56323-217f-4fe1-be14-ec9ad4b7b81a)\"" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw" podUID="c4f56323-217f-4fe1-be14-ec9ad4b7b81a"
Jul 20 18:17:25.554815 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:25.554784    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:25.554815 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:25.554819    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:25.555077 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:25.554892    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 18:17:25.555077 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:25.554961    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm podName:629056e9-18ee-4294-a8ab-42081cff41b4 nodeName:}" failed. No retries permitted until 2026-07-20 18:17:27.554943088 +0000 UTC m=+244.388915454 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7q9t5" (UID: "629056e9-18ee-4294-a8ab-42081cff41b4") : configmap "kube-rbac-proxy" not found
Jul 20 18:17:25.555077 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:25.554988    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 18:17:25.555077 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:25.555045    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls podName:629056e9-18ee-4294-a8ab-42081cff41b4 nodeName:}" failed. No retries permitted until 2026-07-20 18:17:27.555027944 +0000 UTC m=+244.389000312 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7q9t5" (UID: "629056e9-18ee-4294-a8ab-42081cff41b4") : secret "insights-runtime-extractor-tls" not found
Jul 20 18:17:26.491982 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:26.491923    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:26.530427 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:26.530336    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-w9t2q" event={"ID":"f43ff13a-ead5-45d1-9e43-b17ddef43cf5","Type":"ContainerStarted","Data":"d36b3c19dd917825fac39a6933f2f047a6ae1d77278085154c0442cddf71f140"}
Jul 20 18:17:26.530427 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:26.530393    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-w9t2q" event={"ID":"f43ff13a-ead5-45d1-9e43-b17ddef43cf5","Type":"ContainerStarted","Data":"9bb8f8cc51813d8d76c9c536f174ca075462d7f3cb914aba1c1aacc11056012c"}
Jul 20 18:17:26.547842 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:26.547792    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-w9t2q" podStartSLOduration=1.2437184079999999 podStartE2EDuration="3.547775566s" totalImagesPullingTime="2.304057158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:17:26.546980769 +0000 UTC m=+243.380953161" watchObservedRunningTime="2026-07-20 18:17:26.547775566 +0000 UTC m=+243.381747972"
Jul 20 18:17:26.556854 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:26.556822    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:26.630328 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:26.630297    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:26.640959 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:26.640930    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8bf9p_17e7c5f3-421c-4bdb-9e4e-67586149b43d/dns-node-resolver/0.log"
Jul 20 18:17:26.691209 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:26.691181    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:27.570423 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:27.570384    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:27.570423 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:27.570422    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:27.570861 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:27.570509    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 18:17:27.570861 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:27.570573    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm podName:629056e9-18ee-4294-a8ab-42081cff41b4 nodeName:}" failed. No retries permitted until 2026-07-20 18:17:31.570558608 +0000 UTC m=+248.404530974 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7q9t5" (UID: "629056e9-18ee-4294-a8ab-42081cff41b4") : configmap "kube-rbac-proxy" not found
Jul 20 18:17:27.570861 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:27.570604    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 18:17:27.570861 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:27.570656    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls podName:629056e9-18ee-4294-a8ab-42081cff41b4 nodeName:}" failed. No retries permitted until 2026-07-20 18:17:31.570642936 +0000 UTC m=+248.404615301 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7q9t5" (UID: "629056e9-18ee-4294-a8ab-42081cff41b4") : secret "insights-runtime-extractor-tls" not found
Jul 20 18:17:27.637496 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:27.637467    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lkqb9_d5a1608a-991c-4c92-978f-7f00e84fda46/node-ca/0.log"
Jul 20 18:17:27.996046 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:27.996016    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:28.051452 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:28.051405    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:28.237312 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:28.237275    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5n5pt_fb74a816-b89d-4e4d-b322-cc029854d831/serve-healthcheck-canary/0.log"
Jul 20 18:17:28.841305 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:28.841267    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-w9t2q_f43ff13a-ead5-45d1-9e43-b17ddef43cf5/migrator/0.log"
Jul 20 18:17:28.893476 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:28.893428    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:29.039069 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:29.039027    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-w9t2q_f43ff13a-ead5-45d1-9e43-b17ddef43cf5/graceful-termination/0.log"
Jul 20 18:17:29.919497 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:29.919420    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:29.919497 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:29.919472    2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:29.919888 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:29.919804    2569 scope.go:117] "RemoveContainer" containerID="68c0799749f94a8763b08f6846984fab89f363e21501b4c49f1ae432cce82583"
Jul 20 18:17:29.919956 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:29.919938    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-pb2nw_openshift-console-operator(c4f56323-217f-4fe1-be14-ec9ad4b7b81a)\"" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw" podUID="c4f56323-217f-4fe1-be14-ec9ad4b7b81a"
Jul 20 18:17:31.600467 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:31.600435    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:31.600467 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:31.600469    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:31.600993 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:31.600582    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 18:17:31.600993 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:31.600629    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls podName:629056e9-18ee-4294-a8ab-42081cff41b4 nodeName:}" failed. No retries permitted until 2026-07-20 18:17:39.600615783 +0000 UTC m=+256.434588150 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7q9t5" (UID: "629056e9-18ee-4294-a8ab-42081cff41b4") : secret "insights-runtime-extractor-tls" not found
Jul 20 18:17:31.600993 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:31.600575    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 18:17:31.600993 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:31.600673    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm podName:629056e9-18ee-4294-a8ab-42081cff41b4 nodeName:}" failed. No retries permitted until 2026-07-20 18:17:39.600667302 +0000 UTC m=+256.434639667 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7q9t5" (UID: "629056e9-18ee-4294-a8ab-42081cff41b4") : configmap "kube-rbac-proxy" not found
Jul 20 18:17:31.926467 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:31.926380    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:31.995902 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:31.995874    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:32.072773 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:32.072741    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:33.500696 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:33.500662    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:33.552011 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:33.551982    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:33.654210 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:33.654177    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:33.857112 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:33.857027    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:34.060970 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:34.060940    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:34.632154 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:34.632124    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:34.748067 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:34.748035    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:35.052305 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:35.052252    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:35.267620 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:35.267590    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:35.322078 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:35.321976    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:35.631086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:35.630980    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs\") pod \"network-metrics-daemon-g4z7w\" (UID: \"1bae9e54-a7d1-46ba-9bb6-9602c8d53962\") " pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:17:35.631236 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:35.631156    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 20 18:17:35.631236 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:35.631227    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs podName:1bae9e54-a7d1-46ba-9bb6-9602c8d53962 nodeName:}" failed. No retries permitted until 2026-07-20 18:19:37.631212364 +0000 UTC m=+374.465184729 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs") pod "network-metrics-daemon-g4z7w" (UID: "1bae9e54-a7d1-46ba-9bb6-9602c8d53962") : secret "metrics-daemon-secret" not found
Jul 20 18:17:35.655299 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:35.655276    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:35.708687 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:35.708663    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:35.855203 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:35.855172    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:36.463305 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:36.463275    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:36.718360 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:36.718254    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:37.055454 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:37.055419    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:37.251981 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:37.251944    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:37.918214 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:37.918177    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:37.990407 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:37.990375    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:38.483846 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:38.483818    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:39.141209 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:39.141180    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:39.265563 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:39.265522    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:39.476691 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:39.476609    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:39.665342 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:39.665298    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:39.665342 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:39.665341    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:39.665578 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:39.665449    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 18:17:39.665578 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:39.665514    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls podName:629056e9-18ee-4294-a8ab-42081cff41b4 nodeName:}" failed. No retries permitted until 2026-07-20 18:17:55.665497236 +0000 UTC m=+272.499469605 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7q9t5" (UID: "629056e9-18ee-4294-a8ab-42081cff41b4") : secret "insights-runtime-extractor-tls" not found
Jul 20 18:17:39.665944 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:39.665927    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/629056e9-18ee-4294-a8ab-42081cff41b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:39.967500 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:39.967469    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:40.714929 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:40.714896    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:40.752986 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:40.752962    2569 scope.go:117] "RemoveContainer" containerID="68c0799749f94a8763b08f6846984fab89f363e21501b4c49f1ae432cce82583"
Jul 20 18:17:41.082008 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:41.081978    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:41.577174 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:41.577148    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:17:41.577556 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:41.577542    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/1.log"
Jul 20 18:17:41.577615 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:41.577575    2569 generic.go:356] "Generic (PLEG): container finished" podID="c4f56323-217f-4fe1-be14-ec9ad4b7b81a" containerID="986a2b75d8e1ce06bd97f268a053db4c75a4485a63c4c632a973368d1ef60477" exitCode=255
Jul 20 18:17:41.577663 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:41.577618    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw" event={"ID":"c4f56323-217f-4fe1-be14-ec9ad4b7b81a","Type":"ContainerDied","Data":"986a2b75d8e1ce06bd97f268a053db4c75a4485a63c4c632a973368d1ef60477"}
Jul 20 18:17:41.577713 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:41.577664    2569 scope.go:117] "RemoveContainer" containerID="68c0799749f94a8763b08f6846984fab89f363e21501b4c49f1ae432cce82583"
Jul 20 18:17:41.577993 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:41.577972    2569 scope.go:117] "RemoveContainer" containerID="986a2b75d8e1ce06bd97f268a053db4c75a4485a63c4c632a973368d1ef60477"
Jul 20 18:17:41.578190 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:41.578169    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-pb2nw_openshift-console-operator(c4f56323-217f-4fe1-be14-ec9ad4b7b81a)\"" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw" podUID="c4f56323-217f-4fe1-be14-ec9ad4b7b81a"
Jul 20 18:17:42.581390 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:42.581362    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:17:43.385693 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:43.385665    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:49.919370 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:49.919332    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:49.919755 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:49.919383    2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:17:49.919755 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:49.919652    2569 scope.go:117] "RemoveContainer" containerID="986a2b75d8e1ce06bd97f268a053db4c75a4485a63c4c632a973368d1ef60477"
Jul 20 18:17:49.919838 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:17:49.919791    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-pb2nw_openshift-console-operator(c4f56323-217f-4fe1-be14-ec9ad4b7b81a)\"" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw" podUID="c4f56323-217f-4fe1-be14-ec9ad4b7b81a"
Jul 20 18:17:52.423398 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:52.423364    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:52.501827 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:52.501794    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:54.438452 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:54.438419    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:54.508048 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:54.508017    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:55.661262 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:55.661230    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:55.689783 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:55.689758    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:55.692037 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:55.692009    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629056e9-18ee-4294-a8ab-42081cff41b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7q9t5\" (UID: \"629056e9-18ee-4294-a8ab-42081cff41b4\") " pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:55.818526 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:55.818496    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:55.880196 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:55.880167    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9t862"]
Jul 20 18:17:55.882326 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:55.882309    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9t862"
Jul 20 18:17:55.886265 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:55.886243    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-5xzxn\""
Jul 20 18:17:55.886372 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:55.886331    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 20 18:17:55.895777 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:55.895748    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9t862"]
Jul 20 18:17:55.938382 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:55.938273    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:55.965425 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:55.965394    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7q9t5"
Jul 20 18:17:55.992406 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:55.992369    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d9c8c973-4cfe-4a11-bbe0-f2a734ef89a3-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-9t862\" (UID: \"d9c8c973-4cfe-4a11-bbe0-f2a734ef89a3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9t862"
Jul 20 18:17:56.093074 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:56.093022    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d9c8c973-4cfe-4a11-bbe0-f2a734ef89a3-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-9t862\" (UID: \"d9c8c973-4cfe-4a11-bbe0-f2a734ef89a3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9t862"
Jul 20 18:17:56.095425 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:56.095401    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d9c8c973-4cfe-4a11-bbe0-f2a734ef89a3-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-9t862\" (UID: \"d9c8c973-4cfe-4a11-bbe0-f2a734ef89a3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9t862"
Jul 20 18:17:56.101705 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:56.101678    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7q9t5"]
Jul 20 18:17:56.105592 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:17:56.105567    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod629056e9_18ee_4294_a8ab_42081cff41b4.slice/crio-66e7afce23baed4de5dc38060c1a15f747a2f0620c55a3f00bf6b25b01c53060 WatchSource:0}: Error finding container 66e7afce23baed4de5dc38060c1a15f747a2f0620c55a3f00bf6b25b01c53060: Status 404 returned error can't find the container with id 66e7afce23baed4de5dc38060c1a15f747a2f0620c55a3f00bf6b25b01c53060
Jul 20 18:17:56.118188 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:56.118165    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:17:56.191337 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:56.191242    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9t862"
Jul 20 18:17:56.341826 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:56.341793    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9t862"]
Jul 20 18:17:56.346325 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:17:56.346302    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9c8c973_4cfe_4a11_bbe0_f2a734ef89a3.slice/crio-715db2229cb5492dbecd4d68eab025f86d2ec2ae3669511ebf09a10e323e6d9c WatchSource:0}: Error finding container 715db2229cb5492dbecd4d68eab025f86d2ec2ae3669511ebf09a10e323e6d9c: Status 404 returned error can't find the container with id 715db2229cb5492dbecd4d68eab025f86d2ec2ae3669511ebf09a10e323e6d9c
Jul 20 18:17:56.626357 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:56.626317    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9t862" event={"ID":"d9c8c973-4cfe-4a11-bbe0-f2a734ef89a3","Type":"ContainerStarted","Data":"715db2229cb5492dbecd4d68eab025f86d2ec2ae3669511ebf09a10e323e6d9c"}
Jul 20 18:17:56.628019 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:17:56.627941    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7q9t5" event={"ID":"629056e9-18ee-4294-a8ab-42081cff41b4","Type":"ContainerStarted","Data":"66e7afce23baed4de5dc38060c1a15f747a2f0620c55a3f00bf6b25b01c53060"}
Jul 20 18:18:01.646855 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:01.646803    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7q9t5" event={"ID":"629056e9-18ee-4294-a8ab-42081cff41b4","Type":"ContainerStarted","Data":"6d136e7dc5ea083de1d7ba1b36c4d4788599bdeffb1bf3fbadeba9ed6acf61a1"}
Jul 20 18:18:01.648839 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:01.648780    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9t862" event={"ID":"d9c8c973-4cfe-4a11-bbe0-f2a734ef89a3","Type":"ContainerStarted","Data":"2a1e0e7b07766059bfbfde50c75ad5ccc33496dfdcec535c4d04606839928f30"}
Jul 20 18:18:01.649337 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:01.649302    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9t862"
Jul 20 18:18:01.655010 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:01.654988    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9t862"
Jul 20 18:18:01.667909 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:01.667794    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-9t862" podStartSLOduration=2.017838668 podStartE2EDuration="6.667775621s" totalImagesPullingTime="4.649936953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:18:01.666268522 +0000 UTC m=+278.500240910" watchObservedRunningTime="2026-07-20 18:18:01.667775621 +0000 UTC m=+278.501748000"
Jul 20 18:18:01.904424 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:01.904352    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:02.015427 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:02.015396    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:02.187708 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:02.187629    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:04.473204 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:04.473148    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:04.660135 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:04.660035    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7q9t5" event={"ID":"629056e9-18ee-4294-a8ab-42081cff41b4","Type":"ContainerStarted","Data":"bde2204d5834338ea1f5f21f7725cf5af598a2125f7cd3913e2d85cb6df5c7ee"}
Jul 20 18:18:04.752904 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:04.752878    2569 scope.go:117] "RemoveContainer" containerID="986a2b75d8e1ce06bd97f268a053db4c75a4485a63c4c632a973368d1ef60477"
Jul 20 18:18:05.534219 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.534184    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-tvt44"]
Jul 20 18:18:05.536447 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.536424    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-tvt44"
Jul 20 18:18:05.540090 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.540034    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 20 18:18:05.540225 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.540075    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-2h5ck\""
Jul 20 18:18:05.540486 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.540469    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 20 18:18:05.553543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.553482    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-tvt44"]
Jul 20 18:18:05.566601 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.566556    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:05.662888 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.662843    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:05.665338 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.665304    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:18:05.665462 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.665377    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw" event={"ID":"c4f56323-217f-4fe1-be14-ec9ad4b7b81a","Type":"ContainerStarted","Data":"04d7629c0877fe5ac9f9ec1d488782e3948e0fb5344a82ed75cc1fd649d6a1e6"}
Jul 20 18:18:05.665712 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.665659    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:18:05.671239 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.671211    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw"
Jul 20 18:18:05.677617 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.677594    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pxb8\" (UniqueName: \"kubernetes.io/projected/ad8acde9-8e96-4934-be94-7c928abaa587-kube-api-access-2pxb8\") pod \"downloads-5bc67dcd46-tvt44\" (UID: \"ad8acde9-8e96-4934-be94-7c928abaa587\") " pod="openshift-console/downloads-5bc67dcd46-tvt44"
Jul 20 18:18:05.686409 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.686361    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-pb2nw" podStartSLOduration=44.074803138 podStartE2EDuration="46.686344502s" totalImagesPullingTime="2.611541364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:18:05.68533237 +0000 UTC m=+282.519304749" watchObservedRunningTime="2026-07-20 18:18:05.686344502 +0000 UTC m=+282.520316889"
Jul 20 18:18:05.778408 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.778372    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pxb8\" (UniqueName: \"kubernetes.io/projected/ad8acde9-8e96-4934-be94-7c928abaa587-kube-api-access-2pxb8\") pod \"downloads-5bc67dcd46-tvt44\" (UID: \"ad8acde9-8e96-4934-be94-7c928abaa587\") " pod="openshift-console/downloads-5bc67dcd46-tvt44"
Jul 20 18:18:05.790509 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.790425    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pxb8\" (UniqueName: \"kubernetes.io/projected/ad8acde9-8e96-4934-be94-7c928abaa587-kube-api-access-2pxb8\") pod \"downloads-5bc67dcd46-tvt44\" (UID: \"ad8acde9-8e96-4934-be94-7c928abaa587\") " pod="openshift-console/downloads-5bc67dcd46-tvt44"
Jul 20 18:18:05.830259 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.830221    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:05.846135 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:05.846102    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-tvt44"
Jul 20 18:18:06.008866 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.008836    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-tvt44"]
Jul 20 18:18:06.010245 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:18:06.010217    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad8acde9_8e96_4934_be94_7c928abaa587.slice/crio-ea4f0afa0fecfb439dcba607e03360fbbded1003be2767dc70feeaaa547e5db2 WatchSource:0}: Error finding container ea4f0afa0fecfb439dcba607e03360fbbded1003be2767dc70feeaaa547e5db2: Status 404 returned error can't find the container with id ea4f0afa0fecfb439dcba607e03360fbbded1003be2767dc70feeaaa547e5db2
Jul 20 18:18:06.049670 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.049588    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:06.188221 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.188192    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:06.302850 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.302772    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:06.311937 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:18:06.311899    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md" podUID="6fe2898d-c51c-4fe4-a2c5-c554e9e499be"
Jul 20 18:18:06.312102 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:18:06.311899    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-85nj6" podUID="d223e041-067c-49a8-a19c-b4879963bc0e"
Jul 20 18:18:06.414231 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.414194    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:06.429106 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.429079    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-299wt"]
Jul 20 18:18:06.434681 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.434654    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.437955 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.437931    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5lp4n\""
Jul 20 18:18:06.438092 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.437975    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 20 18:18:06.438783 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.438768    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 20 18:18:06.439095 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.439071    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 20 18:18:06.439207 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.439103    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 20 18:18:06.439207 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.439112    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 20 18:18:06.439207 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.439107    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 20 18:18:06.510084 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.510040    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:06.586289 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.586209    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7d6e0182-444e-4a6c-bf6a-c76b947418d6-node-exporter-wtmp\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.586656 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.586289    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d6e0182-444e-4a6c-bf6a-c76b947418d6-sys\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.586656 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.586397    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7d6e0182-444e-4a6c-bf6a-c76b947418d6-root\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.586656 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.586437    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7d6e0182-444e-4a6c-bf6a-c76b947418d6-node-exporter-tls\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.586656 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.586462    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnchw\" (UniqueName: \"kubernetes.io/projected/7d6e0182-444e-4a6c-bf6a-c76b947418d6-kube-api-access-lnchw\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.586656 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.586525    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7d6e0182-444e-4a6c-bf6a-c76b947418d6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.586656 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.586578    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7d6e0182-444e-4a6c-bf6a-c76b947418d6-node-exporter-textfile\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.586656 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.586602    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d6e0182-444e-4a6c-bf6a-c76b947418d6-metrics-client-ca\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.586656 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.586644    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7d6e0182-444e-4a6c-bf6a-c76b947418d6-node-exporter-accelerators-collector-config\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.670242 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.670210    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-85nj6"
Jul 20 18:18:06.670412 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.670251    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:18:06.670412 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.670244    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-tvt44" event={"ID":"ad8acde9-8e96-4934-be94-7c928abaa587","Type":"ContainerStarted","Data":"ea4f0afa0fecfb439dcba607e03360fbbded1003be2767dc70feeaaa547e5db2"}
Jul 20 18:18:06.681021 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.680982    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:06.687481 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.687447    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7d6e0182-444e-4a6c-bf6a-c76b947418d6-root\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.687481 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.687477    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7d6e0182-444e-4a6c-bf6a-c76b947418d6-node-exporter-tls\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.687677 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.687505    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnchw\" (UniqueName: \"kubernetes.io/projected/7d6e0182-444e-4a6c-bf6a-c76b947418d6-kube-api-access-lnchw\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.687677 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.687533    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7d6e0182-444e-4a6c-bf6a-c76b947418d6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.687677 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.687554    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7d6e0182-444e-4a6c-bf6a-c76b947418d6-root\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.687677 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.687576    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7d6e0182-444e-4a6c-bf6a-c76b947418d6-node-exporter-textfile\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.687677 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.687599    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d6e0182-444e-4a6c-bf6a-c76b947418d6-metrics-client-ca\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.687677 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.687651    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7d6e0182-444e-4a6c-bf6a-c76b947418d6-node-exporter-accelerators-collector-config\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.687677 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.687676    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7d6e0182-444e-4a6c-bf6a-c76b947418d6-node-exporter-wtmp\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.688028 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.687800    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7d6e0182-444e-4a6c-bf6a-c76b947418d6-node-exporter-wtmp\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.688028 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.687907    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d6e0182-444e-4a6c-bf6a-c76b947418d6-sys\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.688028 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.687983    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7d6e0182-444e-4a6c-bf6a-c76b947418d6-node-exporter-textfile\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.688214 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.688028    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d6e0182-444e-4a6c-bf6a-c76b947418d6-sys\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.688325 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.688301    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7d6e0182-444e-4a6c-bf6a-c76b947418d6-node-exporter-accelerators-collector-config\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.688404 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.688303    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d6e0182-444e-4a6c-bf6a-c76b947418d6-metrics-client-ca\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.690459 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.690414    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7d6e0182-444e-4a6c-bf6a-c76b947418d6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.690700 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.690683    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7d6e0182-444e-4a6c-bf6a-c76b947418d6-node-exporter-tls\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.708521 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.708460    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnchw\" (UniqueName: \"kubernetes.io/projected/7d6e0182-444e-4a6c-bf6a-c76b947418d6-kube-api-access-lnchw\") pod \"node-exporter-299wt\" (UID: \"7d6e0182-444e-4a6c-bf6a-c76b947418d6\") " pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.746528 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.746496    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-299wt"
Jul 20 18:18:06.778587 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.778530    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:06.942973 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:06.942884    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:07.044927 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.044895    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:07.216416 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.216341    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:07.317268 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.317237    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:07.417042 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.417006    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:07.467181 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.467106    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 18:18:07.479296 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.479264    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.481907 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.481883    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 20 18:18:07.481907 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.481883    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 20 18:18:07.482679 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.482654    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 20 18:18:07.482771 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.482694    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 20 18:18:07.483101 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.483008    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 20 18:18:07.483101 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.483039    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 20 18:18:07.483101 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.483080    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-cbxld\""
Jul 20 18:18:07.483338 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.483079    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 20 18:18:07.483338 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.483012    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 20 18:18:07.483449 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.483379    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 20 18:18:07.488270 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.488251    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 18:18:07.516646 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.516611    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:07.597358 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.597294    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-config-volume\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.597358 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.597353    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4234579d-ed4a-4e12-8afd-881271427a53-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.597849 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.597420    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.597849 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.597477    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgczj\" (UniqueName: \"kubernetes.io/projected/4234579d-ed4a-4e12-8afd-881271427a53-kube-api-access-dgczj\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.597849 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.597521    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.597849 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.597539    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.597849 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.597575    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4234579d-ed4a-4e12-8afd-881271427a53-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.597849 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.597595    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.597849 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.597676    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-web-config\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.597849 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.597735    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4234579d-ed4a-4e12-8afd-881271427a53-config-out\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.597849 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.597772    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.597849 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.597798    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4234579d-ed4a-4e12-8afd-881271427a53-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.597849 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.597835    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4234579d-ed4a-4e12-8afd-881271427a53-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.698804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.698765    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4234579d-ed4a-4e12-8afd-881271427a53-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.698979 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.698814    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.698979 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.698882    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-web-config\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.698979 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.698925    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4234579d-ed4a-4e12-8afd-881271427a53-config-out\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.698979 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.698962    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.699211 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.698987    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4234579d-ed4a-4e12-8afd-881271427a53-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.699211 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.699024    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4234579d-ed4a-4e12-8afd-881271427a53-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.699211 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.699149    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-config-volume\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.699211 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.699173    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4234579d-ed4a-4e12-8afd-881271427a53-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.699211 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.699196    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.699463 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.699225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgczj\" (UniqueName: \"kubernetes.io/projected/4234579d-ed4a-4e12-8afd-881271427a53-kube-api-access-dgczj\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.699463 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.699255    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.699463 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.699281    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.701460 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:18:07.699902    2569 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 20 18:18:07.701460 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:18:07.699969    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-main-tls podName:4234579d-ed4a-4e12-8afd-881271427a53 nodeName:}" failed. No retries permitted until 2026-07-20 18:18:08.199948172 +0000 UTC m=+285.033920550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "4234579d-ed4a-4e12-8afd-881271427a53") : secret "alertmanager-main-tls" not found
Jul 20 18:18:07.701460 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.700267    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4234579d-ed4a-4e12-8afd-881271427a53-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.701460 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.700894    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4234579d-ed4a-4e12-8afd-881271427a53-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.701460 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.700907    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4234579d-ed4a-4e12-8afd-881271427a53-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.702373 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.702316    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4234579d-ed4a-4e12-8afd-881271427a53-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.702473 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.702381    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4234579d-ed4a-4e12-8afd-881271427a53-config-out\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.702537 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.702500    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-web-config\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.703072 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.703033    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.703165 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.703097    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.703165 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.703153    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-config-volume\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.703409 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.703381    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.704324 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.704283    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.717897 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:07.717830    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgczj\" (UniqueName: \"kubernetes.io/projected/4234579d-ed4a-4e12-8afd-881271427a53-kube-api-access-dgczj\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:07.867695 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:18:07.867649    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d6e0182_444e_4a6c_bf6a_c76b947418d6.slice/crio-49089ba0f718539b1c9433fe0cd06a0029068cf6217520766af4c8c6ba5c2101 WatchSource:0}: Error finding container 49089ba0f718539b1c9433fe0cd06a0029068cf6217520766af4c8c6ba5c2101: Status 404 returned error can't find the container with id 49089ba0f718539b1c9433fe0cd06a0029068cf6217520766af4c8c6ba5c2101
Jul 20 18:18:08.067341 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.067307    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:08.134545 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.134509    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:08.203035 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.202996    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:08.205434 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.205398    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:08.390911 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.390828    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:18:08.463738 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.463656    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-564d88c9cb-dm42m"]
Jul 20 18:18:08.466880 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.466859    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.471302 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.471278    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 20 18:18:08.471859 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.471552    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 20 18:18:08.471859 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.471560    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 20 18:18:08.471859 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.471619    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-jxvzg\""
Jul 20 18:18:08.471859 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.471828    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 20 18:18:08.472297 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.472275    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 20 18:18:08.472389 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.472298    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5astkppmcg3ls\""
Jul 20 18:18:08.480950 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.480547    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-564d88c9cb-dm42m"]
Jul 20 18:18:08.545426 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.545398    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 18:18:08.606522 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.606483    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.606961 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.606608    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-thanos-querier-tls\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.606961 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.606682    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.606961 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.606740    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqbnd\" (UniqueName: \"kubernetes.io/projected/1dc72865-f206-4b99-b0b8-6871c20bb36f-kube-api-access-qqbnd\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.606961 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.606798    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.606961 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.606851    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.606961 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.606897    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-grpc-tls\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.606961 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.606922    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dc72865-f206-4b99-b0b8-6871c20bb36f-metrics-client-ca\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.680157 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.680073    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7q9t5" event={"ID":"629056e9-18ee-4294-a8ab-42081cff41b4","Type":"ContainerStarted","Data":"38e655081bc8a0009507bb47c805e553eb0a08cc86a71c52954c8cb571453e40"}
Jul 20 18:18:08.681400 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.681377    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-299wt" event={"ID":"7d6e0182-444e-4a6c-bf6a-c76b947418d6","Type":"ContainerStarted","Data":"49089ba0f718539b1c9433fe0cd06a0029068cf6217520766af4c8c6ba5c2101"}
Jul 20 18:18:08.701795 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.701742    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7q9t5" podStartSLOduration=34.014202897 podStartE2EDuration="45.701725144s" totalImagesPullingTime="11.687522247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 18:17:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:18:08.700224606 +0000 UTC m=+285.534197015" watchObservedRunningTime="2026-07-20 18:18:08.701725144 +0000 UTC m=+285.535697531"
Jul 20 18:18:08.707830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.707793    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.707968 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.707847    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-grpc-tls\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.707968 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.707871    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dc72865-f206-4b99-b0b8-6871c20bb36f-metrics-client-ca\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.707968 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.707900    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.707968 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.707952    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-thanos-querier-tls\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.708199 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.708015    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.708199 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.708081    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqbnd\" (UniqueName: \"kubernetes.io/projected/1dc72865-f206-4b99-b0b8-6871c20bb36f-kube-api-access-qqbnd\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.708199 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.708118    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.708716 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.708661    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dc72865-f206-4b99-b0b8-6871c20bb36f-metrics-client-ca\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.711006 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.710981    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.711149 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.711003    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.711149 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.711009    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-grpc-tls\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.711149 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.711035    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.711277 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.711167    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-thanos-querier-tls\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.711277 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.711178    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1dc72865-f206-4b99-b0b8-6871c20bb36f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.715075 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:18:08.715038    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4234579d_ed4a_4e12_8afd_881271427a53.slice/crio-296b517aabd8baffcf4e5a6b938e5771dfc82baeeb5decf0561968074880351e WatchSource:0}: Error finding container 296b517aabd8baffcf4e5a6b938e5771dfc82baeeb5decf0561968074880351e: Status 404 returned error can't find the container with id 296b517aabd8baffcf4e5a6b938e5771dfc82baeeb5decf0561968074880351e
Jul 20 18:18:08.718944 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.718927    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqbnd\" (UniqueName: \"kubernetes.io/projected/1dc72865-f206-4b99-b0b8-6871c20bb36f-kube-api-access-qqbnd\") pod \"thanos-querier-564d88c9cb-dm42m\" (UID: \"1dc72865-f206-4b99-b0b8-6871c20bb36f\") " pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.779248 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.779225    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:08.911793 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:08.911761    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-564d88c9cb-dm42m"]
Jul 20 18:18:08.960189 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:18:08.960105    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dc72865_f206_4b99_b0b8_6871c20bb36f.slice/crio-916c258a008dfa1d2928d7461f2b599ec6421552034a64c59a973ae6e5981ec5 WatchSource:0}: Error finding container 916c258a008dfa1d2928d7461f2b599ec6421552034a64c59a973ae6e5981ec5: Status 404 returned error can't find the container with id 916c258a008dfa1d2928d7461f2b599ec6421552034a64c59a973ae6e5981ec5
Jul 20 18:18:09.338175 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.338123    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:09.437416 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.437376    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:09.517140 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.517106    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:18:09.517317 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.517171    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:18:09.520108 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.520077    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d223e041-067c-49a8-a19c-b4879963bc0e-metrics-tls\") pod \"dns-default-85nj6\" (UID: \"d223e041-067c-49a8-a19c-b4879963bc0e\") " pod="openshift-dns/dns-default-85nj6"
Jul 20 18:18:09.520461 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.520437    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls\") pod \"image-registry-5875d48c7b-kl9md\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") " pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:18:09.550079 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.550032    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:09.654416 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.654282    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:09.673745 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.673702    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pzczt\""
Jul 20 18:18:09.674108 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.674079    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wr2zv\""
Jul 20 18:18:09.682048 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.682017    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-85nj6"
Jul 20 18:18:09.682231 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.682107    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:18:09.691371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.691341    2569 generic.go:356] "Generic (PLEG): container finished" podID="7d6e0182-444e-4a6c-bf6a-c76b947418d6" containerID="1faed042046039ac997223129de84ae5646ad4318c06831ebf74eee8ae702a10" exitCode=0
Jul 20 18:18:09.692136 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.692105    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-299wt" event={"ID":"7d6e0182-444e-4a6c-bf6a-c76b947418d6","Type":"ContainerDied","Data":"1faed042046039ac997223129de84ae5646ad4318c06831ebf74eee8ae702a10"}
Jul 20 18:18:09.693475 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.693450    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4234579d-ed4a-4e12-8afd-881271427a53","Type":"ContainerStarted","Data":"296b517aabd8baffcf4e5a6b938e5771dfc82baeeb5decf0561968074880351e"}
Jul 20 18:18:09.695316 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.695285    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m" event={"ID":"1dc72865-f206-4b99-b0b8-6871c20bb36f","Type":"ContainerStarted","Data":"916c258a008dfa1d2928d7461f2b599ec6421552034a64c59a973ae6e5981ec5"}
Jul 20 18:18:09.883112 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.882301    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5875d48c7b-kl9md"]
Jul 20 18:18:09.892559 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:18:09.892525    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fe2898d_c51c_4fe4_a2c5_c554e9e499be.slice/crio-60928e71f9c33fad9e8ba87548b0ddd4efd1e3580ea9541af90ffae092436f61 WatchSource:0}: Error finding container 60928e71f9c33fad9e8ba87548b0ddd4efd1e3580ea9541af90ffae092436f61: Status 404 returned error can't find the container with id 60928e71f9c33fad9e8ba87548b0ddd4efd1e3580ea9541af90ffae092436f61
Jul 20 18:18:09.896330 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:09.896241    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-85nj6"]
Jul 20 18:18:09.900496 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:18:09.900254    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd223e041_067c_49a8_a19c_b4879963bc0e.slice/crio-f58378b517581fda250cea0b914a592c6c9e0d2cfbaddc2991889a168396cba2 WatchSource:0}: Error finding container f58378b517581fda250cea0b914a592c6c9e0d2cfbaddc2991889a168396cba2: Status 404 returned error can't find the container with id f58378b517581fda250cea0b914a592c6c9e0d2cfbaddc2991889a168396cba2
Jul 20 18:18:10.706117 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:10.705811    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-299wt" event={"ID":"7d6e0182-444e-4a6c-bf6a-c76b947418d6","Type":"ContainerStarted","Data":"1d0e3695aae3e65a0120007f87bfe87eff244a0a172332757580c640fb2e17bf"}
Jul 20 18:18:10.706117 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:10.705873    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-299wt" event={"ID":"7d6e0182-444e-4a6c-bf6a-c76b947418d6","Type":"ContainerStarted","Data":"cd66e36516262d1f9c491d5c3457b5e7a63f0a9e595606fa91c31b3aca2dd742"}
Jul 20 18:18:10.707799 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:10.707769    2569 generic.go:356] "Generic (PLEG): container finished" podID="4234579d-ed4a-4e12-8afd-881271427a53" containerID="f0ee788207f6d5632e7216c518742da717d886f9ab394a5bb4eeb278babd44ff" exitCode=0
Jul 20 18:18:10.707985 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:10.707864    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4234579d-ed4a-4e12-8afd-881271427a53","Type":"ContainerDied","Data":"f0ee788207f6d5632e7216c518742da717d886f9ab394a5bb4eeb278babd44ff"}
Jul 20 18:18:10.711615 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:10.710982    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md" event={"ID":"6fe2898d-c51c-4fe4-a2c5-c554e9e499be","Type":"ContainerStarted","Data":"9cb6ed599d305d3d73e3f0b5397f17a57c0004ba5c8826f2d2cd433e2b96e4cb"}
Jul 20 18:18:10.711615 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:10.711027    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md" event={"ID":"6fe2898d-c51c-4fe4-a2c5-c554e9e499be","Type":"ContainerStarted","Data":"60928e71f9c33fad9e8ba87548b0ddd4efd1e3580ea9541af90ffae092436f61"}
Jul 20 18:18:10.711615 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:10.711122    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:18:10.712584 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:10.712557    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-85nj6" event={"ID":"d223e041-067c-49a8-a19c-b4879963bc0e","Type":"ContainerStarted","Data":"f58378b517581fda250cea0b914a592c6c9e0d2cfbaddc2991889a168396cba2"}
Jul 20 18:18:10.728484 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:10.728426    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-299wt" podStartSLOduration=3.7842139 podStartE2EDuration="4.72840514s" totalImagesPullingTime="871.328292ms" totalInitContainerRuntime="72.862948ms" isStatefulPod=true podCreationTimestamp="2026-07-20 18:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:18:10.727152715 +0000 UTC m=+287.561125104" watchObservedRunningTime="2026-07-20 18:18:10.72840514 +0000 UTC m=+287.562377529"
Jul 20 18:18:10.781119 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:10.780832    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md" podStartSLOduration=286.780808085 podStartE2EDuration="4m46.780808085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:13:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:18:10.779582908 +0000 UTC m=+287.613555334" watchObservedRunningTime="2026-07-20 18:18:10.780808085 +0000 UTC m=+287.614780474"
Jul 20 18:18:11.125542 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.125473    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:11.130458 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.130398    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-m22pt"]
Jul 20 18:18:11.133747 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.133726    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-m22pt"
Jul 20 18:18:11.136467 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.136442    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-q5hmt\""
Jul 20 18:18:11.136636 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.136447    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 20 18:18:11.150559 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.150521    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-m22pt"]
Jul 20 18:18:11.218879 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.218845    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:11.233177 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.233146    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxl4s\" (UniqueName: \"kubernetes.io/projected/c556e218-7633-4bd3-91c3-f1979da2efec-kube-api-access-hxl4s\") pod \"monitoring-plugin-7b7b7bc8bf-m22pt\" (UID: \"c556e218-7633-4bd3-91c3-f1979da2efec\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-m22pt"
Jul 20 18:18:11.233357 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.233199    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c556e218-7633-4bd3-91c3-f1979da2efec-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-m22pt\" (UID: \"c556e218-7633-4bd3-91c3-f1979da2efec\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-m22pt"
Jul 20 18:18:11.334539 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.334503    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c556e218-7633-4bd3-91c3-f1979da2efec-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-m22pt\" (UID: \"c556e218-7633-4bd3-91c3-f1979da2efec\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-m22pt"
Jul 20 18:18:11.334758 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.334637    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxl4s\" (UniqueName: \"kubernetes.io/projected/c556e218-7633-4bd3-91c3-f1979da2efec-kube-api-access-hxl4s\") pod \"monitoring-plugin-7b7b7bc8bf-m22pt\" (UID: \"c556e218-7633-4bd3-91c3-f1979da2efec\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-m22pt"
Jul 20 18:18:11.337463 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.337434    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c556e218-7633-4bd3-91c3-f1979da2efec-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-m22pt\" (UID: \"c556e218-7633-4bd3-91c3-f1979da2efec\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-m22pt"
Jul 20 18:18:11.348201 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.348146    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxl4s\" (UniqueName: \"kubernetes.io/projected/c556e218-7633-4bd3-91c3-f1979da2efec-kube-api-access-hxl4s\") pod \"monitoring-plugin-7b7b7bc8bf-m22pt\" (UID: \"c556e218-7633-4bd3-91c3-f1979da2efec\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-m22pt"
Jul 20 18:18:11.446228 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.446137    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-m22pt"
Jul 20 18:18:11.673636 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.673567    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-m22pt"]
Jul 20 18:18:11.676915 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:18:11.676885    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc556e218_7633_4bd3_91c3_f1979da2efec.slice/crio-1d221f02fc9c05fb1e0776327bbcef61eba23a4428fe9e46dc5a7119a28e8733 WatchSource:0}: Error finding container 1d221f02fc9c05fb1e0776327bbcef61eba23a4428fe9e46dc5a7119a28e8733: Status 404 returned error can't find the container with id 1d221f02fc9c05fb1e0776327bbcef61eba23a4428fe9e46dc5a7119a28e8733
Jul 20 18:18:11.720454 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.720314    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m" event={"ID":"1dc72865-f206-4b99-b0b8-6871c20bb36f","Type":"ContainerStarted","Data":"1f628dceb41acd9b790f59a5dffee94b7cf428d81e38e4c033817dc8d7cfd647"}
Jul 20 18:18:11.720454 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.720405    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m" event={"ID":"1dc72865-f206-4b99-b0b8-6871c20bb36f","Type":"ContainerStarted","Data":"def58a5c82f6806db44a4aad6c3ebcd7e69fa27cb75129377f223298216ce9a0"}
Jul 20 18:18:11.721961 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:11.721901    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-m22pt" event={"ID":"c556e218-7633-4bd3-91c3-f1979da2efec","Type":"ContainerStarted","Data":"1d221f02fc9c05fb1e0776327bbcef61eba23a4428fe9e46dc5a7119a28e8733"}
Jul 20 18:18:12.659830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.659732    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 18:18:12.660300 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.660276    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:12.664467 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.664444    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.668699 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.668676    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 20 18:18:12.669905 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.669883    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 20 18:18:12.670188 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.670171    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 20 18:18:12.671303 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.670373    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 20 18:18:12.671303 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.670641    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 20 18:18:12.671303 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.670650    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 20 18:18:12.671303 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.670791    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 20 18:18:12.671303 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.670896    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 20 18:18:12.671303 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.670968    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 20 18:18:12.671303 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.671115    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 20 18:18:12.671303 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.671168    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 20 18:18:12.672523 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.671835    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 20 18:18:12.672523 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.672031    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xjfsx\""
Jul 20 18:18:12.672523 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.672219    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 20 18:18:12.674243 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.674215    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 20 18:18:12.674326 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.674247    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-eacu4nhuq66qf\""
Jul 20 18:18:12.679205 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.679182    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 20 18:18:12.691847 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.691822    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 18:18:12.727919 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.727885    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m" event={"ID":"1dc72865-f206-4b99-b0b8-6871c20bb36f","Type":"ContainerStarted","Data":"37e3fd7a6d8dbeabd3048f5570695d63a0d2adc9a00f89110f06a2939691886b"}
Jul 20 18:18:12.729872 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.729845    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-85nj6" event={"ID":"d223e041-067c-49a8-a19c-b4879963bc0e","Type":"ContainerStarted","Data":"05c8cc42ae8e00d8d3c1aee02f5d5a5aba7e861ccddf1c0ed28b645975d44b6b"}
Jul 20 18:18:12.730002 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.729888    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-85nj6" event={"ID":"d223e041-067c-49a8-a19c-b4879963bc0e","Type":"ContainerStarted","Data":"a573146aab597a4820b285d13093d5d68cd023c6fcbe84c2e03190f945dddc66"}
Jul 20 18:18:12.730002 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.729957    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-85nj6"
Jul 20 18:18:12.751303 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.751252    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-85nj6" podStartSLOduration=251.368838742 podStartE2EDuration="4m13.751237585s" totalImagesPullingTime="2.382398843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:13:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:18:12.75064563 +0000 UTC m=+289.584618040" watchObservedRunningTime="2026-07-20 18:18:12.751237585 +0000 UTC m=+289.585209973"
Jul 20 18:18:12.851313 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.851276    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.851492 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.851322    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.851492 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.851349    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.851492 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.851369    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.851643 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.851522    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.851643 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.851565    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.851643 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.851592    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.851779 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.851672    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhpl2\" (UniqueName: \"kubernetes.io/projected/e5ccaaca-901d-476e-aeab-74cd33dbf59a-kube-api-access-hhpl2\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.851779 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.851708    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.851779 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.851730    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.851905 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.851797    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-web-config\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.851905 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.851858    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.851905 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.851886    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.852080 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.851912    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.852080 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.851966    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-config\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.852080 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.852004    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e5ccaaca-901d-476e-aeab-74cd33dbf59a-config-out\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.852080 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.852035    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.852259 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.852099    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e5ccaaca-901d-476e-aeab-74cd33dbf59a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.852259 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.852169    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.852259 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.852198    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.953760 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.953632    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.953760 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.953681    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.953760 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.953708    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.953760 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.953738    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-config\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.953769    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e5ccaaca-901d-476e-aeab-74cd33dbf59a-config-out\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.953795    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.953820    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e5ccaaca-901d-476e-aeab-74cd33dbf59a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.953871    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.953897    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.953928    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.953952    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.953978    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.954005    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.954048    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.954094    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954697 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.954132    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954697 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.954164    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhpl2\" (UniqueName: \"kubernetes.io/projected/e5ccaaca-901d-476e-aeab-74cd33dbf59a-kube-api-access-hhpl2\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954697 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.954201    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954697 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.954226    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954697 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.954270    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-web-config\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.954697 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.954611    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.956031 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.955145    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.956031 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.955716    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.956750 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.956307    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.960169 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.957145    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.960169 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.958735    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.960169 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.959283    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.960169 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.959667    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-web-config\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.960169 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.960128    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.960555 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.960229    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.962243 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.962192    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.962781 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.962740    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-config\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.963273 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.963215    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.963460 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.963340    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.963460 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.963355    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e5ccaaca-901d-476e-aeab-74cd33dbf59a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.964006 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.963883    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.964931 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.964896    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e5ccaaca-901d-476e-aeab-74cd33dbf59a-config-out\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.965290 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.965248    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.965663 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.965642    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.969966 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.969948    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhpl2\" (UniqueName: \"kubernetes.io/projected/e5ccaaca-901d-476e-aeab-74cd33dbf59a-kube-api-access-hhpl2\") pod \"prometheus-k8s-0\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:12.977395 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:12.977306    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:13.162431 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:13.162331    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 18:18:13.535984 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:18:13.535948    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5ccaaca_901d_476e_aeab_74cd33dbf59a.slice/crio-aa16cc170f61a3c20b1f6f123d31c7d5f9e1e5fffd2d5c4961e8f5f8e684a084 WatchSource:0}: Error finding container aa16cc170f61a3c20b1f6f123d31c7d5f9e1e5fffd2d5c4961e8f5f8e684a084: Status 404 returned error can't find the container with id aa16cc170f61a3c20b1f6f123d31c7d5f9e1e5fffd2d5c4961e8f5f8e684a084
Jul 20 18:18:13.734188 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:13.734149    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e5ccaaca-901d-476e-aeab-74cd33dbf59a","Type":"ContainerStarted","Data":"aa16cc170f61a3c20b1f6f123d31c7d5f9e1e5fffd2d5c4961e8f5f8e684a084"}
Jul 20 18:18:14.741527 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.741440    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4234579d-ed4a-4e12-8afd-881271427a53","Type":"ContainerStarted","Data":"9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc"}
Jul 20 18:18:14.741527 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.741504    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4234579d-ed4a-4e12-8afd-881271427a53","Type":"ContainerStarted","Data":"0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197"}
Jul 20 18:18:14.741980 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.741547    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4234579d-ed4a-4e12-8afd-881271427a53","Type":"ContainerStarted","Data":"9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573"}
Jul 20 18:18:14.741980 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.741582    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4234579d-ed4a-4e12-8afd-881271427a53","Type":"ContainerStarted","Data":"dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e"}
Jul 20 18:18:14.741980 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.741615    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4234579d-ed4a-4e12-8afd-881271427a53","Type":"ContainerStarted","Data":"93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363"}
Jul 20 18:18:14.741980 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.741645    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4234579d-ed4a-4e12-8afd-881271427a53","Type":"ContainerStarted","Data":"88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071"}
Jul 20 18:18:14.744300 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.744276    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m" event={"ID":"1dc72865-f206-4b99-b0b8-6871c20bb36f","Type":"ContainerStarted","Data":"31637587ac937f5ad1ac1fb92ecefe235e8330fda51d32397b24f1643a9f771d"}
Jul 20 18:18:14.744482 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.744328    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m" event={"ID":"1dc72865-f206-4b99-b0b8-6871c20bb36f","Type":"ContainerStarted","Data":"ad84fde67247f482b2cc254ca560a0c4aa6485bc91fdc96706aa3842aa6dc360"}
Jul 20 18:18:14.744482 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.744354    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m" event={"ID":"1dc72865-f206-4b99-b0b8-6871c20bb36f","Type":"ContainerStarted","Data":"6180b27d2e10cb2907916c32f12c611477517b2fc71ecbbb128b774560040ec2"}
Jul 20 18:18:14.744482 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.744392    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:14.745851 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.745825    2569 generic.go:356] "Generic (PLEG): container finished" podID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerID="a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b" exitCode=0
Jul 20 18:18:14.745972 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.745919    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e5ccaaca-901d-476e-aeab-74cd33dbf59a","Type":"ContainerDied","Data":"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b"}
Jul 20 18:18:14.748361 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.748332    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-m22pt" event={"ID":"c556e218-7633-4bd3-91c3-f1979da2efec","Type":"ContainerStarted","Data":"de588f98c5e2877cf1cabe08c698656e772f6f0e803ebaad1d56278f61b2f896"}
Jul 20 18:18:14.748587 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.748566    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-m22pt"
Jul 20 18:18:14.753892 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.753872    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-m22pt"
Jul 20 18:18:14.770479 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.770207    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.777068622 podStartE2EDuration="7.770188592s" totalImagesPullingTime="4.97225208s" totalInitContainerRuntime="20.86789ms" isStatefulPod=false podCreationTimestamp="2026-07-20 18:18:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:18:14.768636684 +0000 UTC m=+291.602609108" watchObservedRunningTime="2026-07-20 18:18:14.770188592 +0000 UTC m=+291.604160981"
Jul 20 18:18:14.790077 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.790001    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-m22pt" podStartSLOduration=1.374500837 podStartE2EDuration="3.789979789s" totalImagesPullingTime="2.415478952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:18:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:18:14.788785308 +0000 UTC m=+291.622757697" watchObservedRunningTime="2026-07-20 18:18:14.789979789 +0000 UTC m=+291.623952179"
Jul 20 18:18:14.817887 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:14.817834    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m" podStartSLOduration=1.9085738079999999 podStartE2EDuration="6.817821789s" totalImagesPullingTime="4.909247981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:18:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:18:14.816747073 +0000 UTC m=+291.650719461" watchObservedRunningTime="2026-07-20 18:18:14.817821789 +0000 UTC m=+291.651794176"
Jul 20 18:18:16.455212 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:16.455169    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:16.528403 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:16.528370    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:18.349163 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:18.348902    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79" podUID="ec6b7d20-2ce7-4819-82bf-36a12f08275b" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.9:8000/readyz\": dial tcp 10.133.0.9:8000: connect: connection refused"
Jul 20 18:18:18.775845 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:18.775812    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e5ccaaca-901d-476e-aeab-74cd33dbf59a","Type":"ContainerStarted","Data":"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd"}
Jul 20 18:18:18.776001 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:18.775885    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e5ccaaca-901d-476e-aeab-74cd33dbf59a","Type":"ContainerStarted","Data":"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa"}
Jul 20 18:18:18.776001 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:18.775931    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e5ccaaca-901d-476e-aeab-74cd33dbf59a","Type":"ContainerStarted","Data":"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38"}
Jul 20 18:18:18.776001 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:18.775962    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e5ccaaca-901d-476e-aeab-74cd33dbf59a","Type":"ContainerStarted","Data":"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47"}
Jul 20 18:18:18.776001 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:18.775986    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e5ccaaca-901d-476e-aeab-74cd33dbf59a","Type":"ContainerStarted","Data":"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c"}
Jul 20 18:18:18.776243 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:18.776019    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e5ccaaca-901d-476e-aeab-74cd33dbf59a","Type":"ContainerStarted","Data":"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da"}
Jul 20 18:18:18.777562 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:18.777537    2569 generic.go:356] "Generic (PLEG): container finished" podID="ec6b7d20-2ce7-4819-82bf-36a12f08275b" containerID="7fe67f36d0cc75c45532e0c8cb8d62cd045a811dd005723f22078c4c227300bd" exitCode=1
Jul 20 18:18:18.777670 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:18.777586    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79" event={"ID":"ec6b7d20-2ce7-4819-82bf-36a12f08275b","Type":"ContainerDied","Data":"7fe67f36d0cc75c45532e0c8cb8d62cd045a811dd005723f22078c4c227300bd"}
Jul 20 18:18:18.777670 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:18.777636    2569 scope.go:117] "RemoveContainer" containerID="e44a140409a72ec407f6e757a156180ea1a44e55dee709fabe3ca1940ddb6a8a"
Jul 20 18:18:18.777893 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:18.777879    2569 scope.go:117] "RemoveContainer" containerID="7fe67f36d0cc75c45532e0c8cb8d62cd045a811dd005723f22078c4c227300bd"
Jul 20 18:18:18.778101 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:18:18.778073    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-98b95c4fd-srl79_open-cluster-management-agent-addon(ec6b7d20-2ce7-4819-82bf-36a12f08275b)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79" podUID="ec6b7d20-2ce7-4819-82bf-36a12f08275b"
Jul 20 18:18:18.808170 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:18.808131    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.477519833 podStartE2EDuration="6.808115656s" totalImagesPullingTime="3.298815261s" totalInitContainerRuntime="31.780562ms" isStatefulPod=false podCreationTimestamp="2026-07-20 18:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:18:18.806115168 +0000 UTC m=+295.640087558" watchObservedRunningTime="2026-07-20 18:18:18.808115656 +0000 UTC m=+295.642088043"
Jul 20 18:18:19.638395 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:19.638355    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:19.739321 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:19.739286    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:20.753534 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:18:20.753500    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-g4z7w" podUID="1bae9e54-a7d1-46ba-9bb6-9602c8d53962"
Jul 20 18:18:20.764979 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:20.764952    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-564d88c9cb-dm42m"
Jul 20 18:18:22.736618 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:22.736591    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-85nj6"
Jul 20 18:18:22.977807 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:22.977752    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:18:23.143327 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:23.143284    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:23.245040 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:23.244998    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:23.345197 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:23.345162    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:27.333809 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:27.333771    2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:18:27.581725 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:27.334268    2569 scope.go:117] "RemoveContainer" containerID="7fe67f36d0cc75c45532e0c8cb8d62cd045a811dd005723f22078c4c227300bd"
Jul 20 18:18:27.581725 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:18:27.334440    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-98b95c4fd-srl79_open-cluster-management-agent-addon(ec6b7d20-2ce7-4819-82bf-36a12f08275b)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79" podUID="ec6b7d20-2ce7-4819-82bf-36a12f08275b"
Jul 20 18:18:28.348832 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:28.348789    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:18:28.349273 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:28.349263    2569 scope.go:117] "RemoveContainer" containerID="7fe67f36d0cc75c45532e0c8cb8d62cd045a811dd005723f22078c4c227300bd"
Jul 20 18:18:29.689184 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:29.689148    2569 patch_prober.go:28] interesting pod/image-registry-5875d48c7b-kl9md container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 20 18:18:29.689624 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:29.689208    2569 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md" podUID="6fe2898d-c51c-4fe4-a2c5-c554e9e499be" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 18:18:30.080811 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:30.080791    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:18:30.080947 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:30.080793    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:18:30.091655 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:30.091635    2569 kubelet.go:1648] "Image garbage collection succeeded"
Jul 20 18:18:30.837152 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:30.837114    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-tvt44" event={"ID":"ad8acde9-8e96-4934-be94-7c928abaa587","Type":"ContainerStarted","Data":"b7290931717fa1f2ec30fefdd616faa86471088e2141cafda2970cf2d9f86dd2"}
Jul 20 18:18:30.837633 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:30.837374    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-tvt44"
Jul 20 18:18:30.839436 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:30.839411    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79" event={"ID":"ec6b7d20-2ce7-4819-82bf-36a12f08275b","Type":"ContainerStarted","Data":"1d0530a10a1dc76f9885987aa7e8b4cb4f56de2ed319f8b4f2b78bf841c09432"}
Jul 20 18:18:30.839700 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:30.839679    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:18:30.841034 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:30.841015    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-98b95c4fd-srl79"
Jul 20 18:18:30.841343 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:30.841308    2569 generic.go:356] "Generic (PLEG): container finished" podID="162cbe48-6490-47b2-b881-85d8e8e95e19" containerID="2afe71d3318016d7c9e5125dc9a43cd4f81b44713b5671fc64ebd6325a7f1de9" exitCode=0
Jul 20 18:18:30.841448 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:30.841374    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-t7pmk" event={"ID":"162cbe48-6490-47b2-b881-85d8e8e95e19","Type":"ContainerDied","Data":"2afe71d3318016d7c9e5125dc9a43cd4f81b44713b5671fc64ebd6325a7f1de9"}
Jul 20 18:18:30.841658 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:30.841630    2569 scope.go:117] "RemoveContainer" containerID="2afe71d3318016d7c9e5125dc9a43cd4f81b44713b5671fc64ebd6325a7f1de9"
Jul 20 18:18:30.850370 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:30.850349    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-tvt44"
Jul 20 18:18:30.853999 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:30.853964    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:30.870556 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:30.870503    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-tvt44" podStartSLOduration=1.4680961209999999 podStartE2EDuration="25.87048403s" totalImagesPullingTime="24.402387909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:18:30.867128778 +0000 UTC m=+307.701101166" watchObservedRunningTime="2026-07-20 18:18:30.87048403 +0000 UTC m=+307.704456418"
Jul 20 18:18:31.728209 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:31.728172    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:18:31.848578 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:31.848539    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-t7pmk" event={"ID":"162cbe48-6490-47b2-b881-85d8e8e95e19","Type":"ContainerStarted","Data":"1f63c7ebac14deab31dc63c56c1306fbd033c62b99f25f8dd386cf8d2b3ecaec"}
Jul 20 18:18:31.949570 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:31.949533    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:32.055506 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:32.055472    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:32.155784 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:32.155729    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:32.521343 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:32.521300    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:32.627765 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:32.627732    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:32.753442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:32.753403    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:18:33.485728 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:33.485694    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:33.874743 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:33.874711    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:34.465025 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:34.464992    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:34.567835 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:34.567803    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:40.889242 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:40.889209    2569 generic.go:356] "Generic (PLEG): container finished" podID="0c1d457d-6952-4712-a0f3-a4b769db92fa" containerID="b53e3264fd528ecdbde31548b66e438be4cd5d46315a591488fb244d93bbf03a" exitCode=0
Jul 20 18:18:40.889694 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:40.889270    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd" event={"ID":"0c1d457d-6952-4712-a0f3-a4b769db92fa","Type":"ContainerDied","Data":"b53e3264fd528ecdbde31548b66e438be4cd5d46315a591488fb244d93bbf03a"}
Jul 20 18:18:40.889694 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:40.889599    2569 scope.go:117] "RemoveContainer" containerID="b53e3264fd528ecdbde31548b66e438be4cd5d46315a591488fb244d93bbf03a"
Jul 20 18:18:40.961417 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:40.961392    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:41.064482 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:41.064445    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:41.690195 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:41.690164    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5875d48c7b-kl9md"]
Jul 20 18:18:41.791391 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:41.791364    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:41.890594 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:41.890562    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:41.894554 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:41.894528    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qpzjd" event={"ID":"0c1d457d-6952-4712-a0f3-a4b769db92fa","Type":"ContainerStarted","Data":"68d5e74bb9e1f41771b00ec829e71fecac01a1ae95de6016f43849cf4b1286fd"}
Jul 20 18:18:41.997946 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:41.997916    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:50.032418 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:50.032380    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:50.131886 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:50.131856    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:56.764463 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:56.764428    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:56.861509 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:56.861476    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:58.769264 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:58.769232    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:18:58.842265 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:18:58.842235    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:05.535902 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:05.535850    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:05.633568 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:05.633537    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:05.695613 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:05.695567    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:06.712347 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:06.712309    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md" podUID="6fe2898d-c51c-4fe4-a2c5-c554e9e499be" containerName="registry" containerID="cri-o://9cb6ed599d305d3d73e3f0b5397f17a57c0004ba5c8826f2d2cd433e2b96e4cb" gracePeriod=30
Jul 20 18:19:07.994483 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:07.994446    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:19:07.997148 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:07.997123    2569 generic.go:356] "Generic (PLEG): container finished" podID="6fe2898d-c51c-4fe4-a2c5-c554e9e499be" containerID="9cb6ed599d305d3d73e3f0b5397f17a57c0004ba5c8826f2d2cd433e2b96e4cb" exitCode=0
Jul 20 18:19:07.997254 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:07.997165    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md"
Jul 20 18:19:07.997254 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:07.997203    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md" event={"ID":"6fe2898d-c51c-4fe4-a2c5-c554e9e499be","Type":"ContainerDied","Data":"9cb6ed599d305d3d73e3f0b5397f17a57c0004ba5c8826f2d2cd433e2b96e4cb"}
Jul 20 18:19:07.997322 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:07.997255    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5875d48c7b-kl9md" event={"ID":"6fe2898d-c51c-4fe4-a2c5-c554e9e499be","Type":"ContainerDied","Data":"60928e71f9c33fad9e8ba87548b0ddd4efd1e3580ea9541af90ffae092436f61"}
Jul 20 18:19:07.997322 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:07.997278    2569 scope.go:117] "RemoveContainer" containerID="9cb6ed599d305d3d73e3f0b5397f17a57c0004ba5c8826f2d2cd433e2b96e4cb"
Jul 20 18:19:08.005295 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.005236    2569 scope.go:117] "RemoveContainer" containerID="9cb6ed599d305d3d73e3f0b5397f17a57c0004ba5c8826f2d2cd433e2b96e4cb"
Jul 20 18:19:08.005592 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:19:08.005570    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cb6ed599d305d3d73e3f0b5397f17a57c0004ba5c8826f2d2cd433e2b96e4cb\": container with ID starting with 9cb6ed599d305d3d73e3f0b5397f17a57c0004ba5c8826f2d2cd433e2b96e4cb not found: ID does not exist" containerID="9cb6ed599d305d3d73e3f0b5397f17a57c0004ba5c8826f2d2cd433e2b96e4cb"
Jul 20 18:19:08.005669 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.005600    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cb6ed599d305d3d73e3f0b5397f17a57c0004ba5c8826f2d2cd433e2b96e4cb"} err="failed to get container status \"9cb6ed599d305d3d73e3f0b5397f17a57c0004ba5c8826f2d2cd433e2b96e4cb\": rpc error: code = NotFound desc = could not find container \"9cb6ed599d305d3d73e3f0b5397f17a57c0004ba5c8826f2d2cd433e2b96e4cb\": container with ID starting with 9cb6ed599d305d3d73e3f0b5397f17a57c0004ba5c8826f2d2cd433e2b96e4cb not found: ID does not exist"
Jul 20 18:19:08.065925 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.065834    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-image-registry-private-configuration\") pod \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") "
Jul 20 18:19:08.065925 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.065887    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-installation-pull-secrets\") pod \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") "
Jul 20 18:19:08.066166 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.065928    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-ca-trust-extracted\") pod \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") "
Jul 20 18:19:08.066166 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.065955    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-trusted-ca\") pod \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") "
Jul 20 18:19:08.066166 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.065992    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mn964\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-kube-api-access-mn964\") pod \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") "
Jul 20 18:19:08.066166 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.066045    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-certificates\") pod \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") "
Jul 20 18:19:08.066166 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.066103    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls\") pod \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") "
Jul 20 18:19:08.066166 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.066145    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-bound-sa-token\") pod \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\" (UID: \"6fe2898d-c51c-4fe4-a2c5-c554e9e499be\") "
Jul 20 18:19:08.066442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.066405    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "6fe2898d-c51c-4fe4-a2c5-c554e9e499be" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:19:08.066518 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.066489    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "6fe2898d-c51c-4fe4-a2c5-c554e9e499be" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:19:08.068486 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.068452    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "6fe2898d-c51c-4fe4-a2c5-c554e9e499be" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:19:08.068623 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.068517    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "6fe2898d-c51c-4fe4-a2c5-c554e9e499be" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:08.068623 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.068600    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-kube-api-access-mn964" (OuterVolumeSpecName: "kube-api-access-mn964") pod "6fe2898d-c51c-4fe4-a2c5-c554e9e499be" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be"). InnerVolumeSpecName "kube-api-access-mn964". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:19:08.068827 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.068809    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6fe2898d-c51c-4fe4-a2c5-c554e9e499be" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:19:08.068919 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.068853    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "6fe2898d-c51c-4fe4-a2c5-c554e9e499be" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:08.074756 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.074732    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "6fe2898d-c51c-4fe4-a2c5-c554e9e499be" (UID: "6fe2898d-c51c-4fe4-a2c5-c554e9e499be"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 18:19:08.167643 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.167599    2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-tls\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:08.167643 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.167634    2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-bound-sa-token\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:08.167643 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.167648    2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-image-registry-private-configuration\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:08.167888 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.167660    2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-installation-pull-secrets\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:08.167888 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.167674    2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-ca-trust-extracted\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:08.167888 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.167686    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-trusted-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:08.167888 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.167697    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn964\" (UniqueName: \"kubernetes.io/projected/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-kube-api-access-mn964\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:08.167888 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.167707    2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6fe2898d-c51c-4fe4-a2c5-c554e9e499be-registry-certificates\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:08.318212 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.318136    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5875d48c7b-kl9md"]
Jul 20 18:19:08.321760 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:08.321735    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5875d48c7b-kl9md"]
Jul 20 18:19:09.757877 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:09.757846    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fe2898d-c51c-4fe4-a2c5-c554e9e499be" path="/var/lib/kubelet/pods/6fe2898d-c51c-4fe4-a2c5-c554e9e499be/volumes"
Jul 20 18:19:11.751091 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:11.751043    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:11.852278 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:11.852227    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:12.977837 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:12.977801    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:13.032297 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:13.032272    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:13.047421 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:13.047391    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:19.663669 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:19.663572    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:19.729736 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:19.729700    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:25.634307 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:25.634273    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:26.041950 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:26.041907    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:26.712983 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:26.712947    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 18:19:26.713476 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:26.713231    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="alertmanager" containerID="cri-o://88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071" gracePeriod=120
Jul 20 18:19:26.713476 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:26.713334    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="kube-rbac-proxy-metric" containerID="cri-o://0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197" gracePeriod=120
Jul 20 18:19:26.713476 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:26.713377    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="config-reloader" containerID="cri-o://93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363" gracePeriod=120
Jul 20 18:19:26.713476 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:26.713335    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="kube-rbac-proxy-web" containerID="cri-o://dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e" gracePeriod=120
Jul 20 18:19:26.713661 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:26.713350    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="kube-rbac-proxy" containerID="cri-o://9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573" gracePeriod=120
Jul 20 18:19:26.713661 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:26.713394    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="prom-label-proxy" containerID="cri-o://9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc" gracePeriod=120
Jul 20 18:19:26.760154 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:26.760119    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:26.815388 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:26.815355    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:27.072966 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:27.072926    2569 generic.go:356] "Generic (PLEG): container finished" podID="4234579d-ed4a-4e12-8afd-881271427a53" containerID="9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc" exitCode=0
Jul 20 18:19:27.072966 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:27.072955    2569 generic.go:356] "Generic (PLEG): container finished" podID="4234579d-ed4a-4e12-8afd-881271427a53" containerID="9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573" exitCode=0
Jul 20 18:19:27.072966 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:27.072962    2569 generic.go:356] "Generic (PLEG): container finished" podID="4234579d-ed4a-4e12-8afd-881271427a53" containerID="93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363" exitCode=0
Jul 20 18:19:27.072966 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:27.072969    2569 generic.go:356] "Generic (PLEG): container finished" podID="4234579d-ed4a-4e12-8afd-881271427a53" containerID="88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071" exitCode=0
Jul 20 18:19:27.073291 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:27.072985    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4234579d-ed4a-4e12-8afd-881271427a53","Type":"ContainerDied","Data":"9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc"}
Jul 20 18:19:27.073291 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:27.073045    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4234579d-ed4a-4e12-8afd-881271427a53","Type":"ContainerDied","Data":"9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573"}
Jul 20 18:19:27.073291 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:27.073099    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4234579d-ed4a-4e12-8afd-881271427a53","Type":"ContainerDied","Data":"93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363"}
Jul 20 18:19:27.073291 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:27.073128    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4234579d-ed4a-4e12-8afd-881271427a53","Type":"ContainerDied","Data":"88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071"}
Jul 20 18:19:27.964378 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:27.964344    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.044220 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.044138    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-web-config\") pod \"4234579d-ed4a-4e12-8afd-881271427a53\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") "
Jul 20 18:19:28.044220 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.044177    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4234579d-ed4a-4e12-8afd-881271427a53-metrics-client-ca\") pod \"4234579d-ed4a-4e12-8afd-881271427a53\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") "
Jul 20 18:19:28.044220 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.044206    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy\") pod \"4234579d-ed4a-4e12-8afd-881271427a53\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") "
Jul 20 18:19:28.044480 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.044248    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4234579d-ed4a-4e12-8afd-881271427a53-alertmanager-trusted-ca-bundle\") pod \"4234579d-ed4a-4e12-8afd-881271427a53\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") "
Jul 20 18:19:28.044480 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.044308    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgczj\" (UniqueName: \"kubernetes.io/projected/4234579d-ed4a-4e12-8afd-881271427a53-kube-api-access-dgczj\") pod \"4234579d-ed4a-4e12-8afd-881271427a53\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") "
Jul 20 18:19:28.044480 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.044442    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy-web\") pod \"4234579d-ed4a-4e12-8afd-881271427a53\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") "
Jul 20 18:19:28.044635 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.044513    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4234579d-ed4a-4e12-8afd-881271427a53-config-out\") pod \"4234579d-ed4a-4e12-8afd-881271427a53\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") "
Jul 20 18:19:28.044635 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.044526    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4234579d-ed4a-4e12-8afd-881271427a53-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4234579d-ed4a-4e12-8afd-881271427a53" (UID: "4234579d-ed4a-4e12-8afd-881271427a53"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:19:28.044635 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.044545    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4234579d-ed4a-4e12-8afd-881271427a53-tls-assets\") pod \"4234579d-ed4a-4e12-8afd-881271427a53\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") "
Jul 20 18:19:28.044635 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.044573    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-config-volume\") pod \"4234579d-ed4a-4e12-8afd-881271427a53\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") "
Jul 20 18:19:28.044635 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.044598    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-cluster-tls-config\") pod \"4234579d-ed4a-4e12-8afd-881271427a53\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") "
Jul 20 18:19:28.044635 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.044633    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-main-tls\") pod \"4234579d-ed4a-4e12-8afd-881271427a53\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") "
Jul 20 18:19:28.044935 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.044662    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4234579d-ed4a-4e12-8afd-881271427a53-alertmanager-main-db\") pod \"4234579d-ed4a-4e12-8afd-881271427a53\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") "
Jul 20 18:19:28.044935 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.044683    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4234579d-ed4a-4e12-8afd-881271427a53-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4234579d-ed4a-4e12-8afd-881271427a53" (UID: "4234579d-ed4a-4e12-8afd-881271427a53"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:19:28.044935 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.044697    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4234579d-ed4a-4e12-8afd-881271427a53\" (UID: \"4234579d-ed4a-4e12-8afd-881271427a53\") "
Jul 20 18:19:28.045138 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.044996    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4234579d-ed4a-4e12-8afd-881271427a53-metrics-client-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:28.045138 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.045016    2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4234579d-ed4a-4e12-8afd-881271427a53-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:28.046098 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.046044    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4234579d-ed4a-4e12-8afd-881271427a53-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4234579d-ed4a-4e12-8afd-881271427a53" (UID: "4234579d-ed4a-4e12-8afd-881271427a53"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 18:19:28.047047 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.047012    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4234579d-ed4a-4e12-8afd-881271427a53" (UID: "4234579d-ed4a-4e12-8afd-881271427a53"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:28.047589 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.047556    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4234579d-ed4a-4e12-8afd-881271427a53" (UID: "4234579d-ed4a-4e12-8afd-881271427a53"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:28.047829 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.047801    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-config-volume" (OuterVolumeSpecName: "config-volume") pod "4234579d-ed4a-4e12-8afd-881271427a53" (UID: "4234579d-ed4a-4e12-8afd-881271427a53"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:28.048210 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.048178    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4234579d-ed4a-4e12-8afd-881271427a53-kube-api-access-dgczj" (OuterVolumeSpecName: "kube-api-access-dgczj") pod "4234579d-ed4a-4e12-8afd-881271427a53" (UID: "4234579d-ed4a-4e12-8afd-881271427a53"). InnerVolumeSpecName "kube-api-access-dgczj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:19:28.048731 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.048704    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4234579d-ed4a-4e12-8afd-881271427a53" (UID: "4234579d-ed4a-4e12-8afd-881271427a53"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:28.048830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.048774    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4234579d-ed4a-4e12-8afd-881271427a53-config-out" (OuterVolumeSpecName: "config-out") pod "4234579d-ed4a-4e12-8afd-881271427a53" (UID: "4234579d-ed4a-4e12-8afd-881271427a53"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 18:19:28.049614 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.049594    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4234579d-ed4a-4e12-8afd-881271427a53-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4234579d-ed4a-4e12-8afd-881271427a53" (UID: "4234579d-ed4a-4e12-8afd-881271427a53"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:19:28.049614 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.049598    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4234579d-ed4a-4e12-8afd-881271427a53" (UID: "4234579d-ed4a-4e12-8afd-881271427a53"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:28.053292 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.053263    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4234579d-ed4a-4e12-8afd-881271427a53" (UID: "4234579d-ed4a-4e12-8afd-881271427a53"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:28.058559 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.058537    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-web-config" (OuterVolumeSpecName: "web-config") pod "4234579d-ed4a-4e12-8afd-881271427a53" (UID: "4234579d-ed4a-4e12-8afd-881271427a53"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:28.079832 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.079804    2569 generic.go:356] "Generic (PLEG): container finished" podID="4234579d-ed4a-4e12-8afd-881271427a53" containerID="0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197" exitCode=0
Jul 20 18:19:28.079832 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.079829    2569 generic.go:356] "Generic (PLEG): container finished" podID="4234579d-ed4a-4e12-8afd-881271427a53" containerID="dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e" exitCode=0
Jul 20 18:19:28.080004 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.079858    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4234579d-ed4a-4e12-8afd-881271427a53","Type":"ContainerDied","Data":"0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197"}
Jul 20 18:19:28.080004 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.079903    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.080004 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.079956    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4234579d-ed4a-4e12-8afd-881271427a53","Type":"ContainerDied","Data":"dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e"}
Jul 20 18:19:28.080004 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.079983    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4234579d-ed4a-4e12-8afd-881271427a53","Type":"ContainerDied","Data":"296b517aabd8baffcf4e5a6b938e5771dfc82baeeb5decf0561968074880351e"}
Jul 20 18:19:28.080149 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.080012    2569 scope.go:117] "RemoveContainer" containerID="9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc"
Jul 20 18:19:28.087897 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.087880    2569 scope.go:117] "RemoveContainer" containerID="0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197"
Jul 20 18:19:28.095443 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.095426    2569 scope.go:117] "RemoveContainer" containerID="9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573"
Jul 20 18:19:28.102257 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.102234    2569 scope.go:117] "RemoveContainer" containerID="dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e"
Jul 20 18:19:28.104994 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.104973    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 18:19:28.110147 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.110124    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 18:19:28.110855 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.110841    2569 scope.go:117] "RemoveContainer" containerID="93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363"
Jul 20 18:19:28.118209 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.118190    2569 scope.go:117] "RemoveContainer" containerID="88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071"
Jul 20 18:19:28.125497 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.125478    2569 scope.go:117] "RemoveContainer" containerID="f0ee788207f6d5632e7216c518742da717d886f9ab394a5bb4eeb278babd44ff"
Jul 20 18:19:28.133198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.133181    2569 scope.go:117] "RemoveContainer" containerID="9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc"
Jul 20 18:19:28.133519 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:19:28.133497    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc\": container with ID starting with 9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc not found: ID does not exist" containerID="9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc"
Jul 20 18:19:28.133606 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.133527    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc"} err="failed to get container status \"9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc\": rpc error: code = NotFound desc = could not find container \"9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc\": container with ID starting with 9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc not found: ID does not exist"
Jul 20 18:19:28.133606 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.133547    2569 scope.go:117] "RemoveContainer" containerID="0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197"
Jul 20 18:19:28.133835 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:19:28.133817    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197\": container with ID starting with 0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197 not found: ID does not exist" containerID="0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197"
Jul 20 18:19:28.133881 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.133840    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197"} err="failed to get container status \"0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197\": rpc error: code = NotFound desc = could not find container \"0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197\": container with ID starting with 0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197 not found: ID does not exist"
Jul 20 18:19:28.133881 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.133856    2569 scope.go:117] "RemoveContainer" containerID="9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573"
Jul 20 18:19:28.134099 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:19:28.134082    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573\": container with ID starting with 9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573 not found: ID does not exist" containerID="9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573"
Jul 20 18:19:28.134153 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.134102    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573"} err="failed to get container status \"9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573\": rpc error: code = NotFound desc = could not find container \"9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573\": container with ID starting with 9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573 not found: ID does not exist"
Jul 20 18:19:28.134153 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.134116    2569 scope.go:117] "RemoveContainer" containerID="dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e"
Jul 20 18:19:28.134309 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:19:28.134287    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e\": container with ID starting with dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e not found: ID does not exist" containerID="dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e"
Jul 20 18:19:28.134401 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.134311    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e"} err="failed to get container status \"dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e\": rpc error: code = NotFound desc = could not find container \"dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e\": container with ID starting with dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e not found: ID does not exist"
Jul 20 18:19:28.134401 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.134323    2569 scope.go:117] "RemoveContainer" containerID="93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363"
Jul 20 18:19:28.134541 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:19:28.134524    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363\": container with ID starting with 93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363 not found: ID does not exist" containerID="93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363"
Jul 20 18:19:28.134583 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.134545    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363"} err="failed to get container status \"93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363\": rpc error: code = NotFound desc = could not find container \"93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363\": container with ID starting with 93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363 not found: ID does not exist"
Jul 20 18:19:28.134583 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.134559    2569 scope.go:117] "RemoveContainer" containerID="88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071"
Jul 20 18:19:28.134807 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:19:28.134793    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071\": container with ID starting with 88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071 not found: ID does not exist" containerID="88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071"
Jul 20 18:19:28.134862 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.134809    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071"} err="failed to get container status \"88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071\": rpc error: code = NotFound desc = could not find container \"88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071\": container with ID starting with 88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071 not found: ID does not exist"
Jul 20 18:19:28.134862 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.134820    2569 scope.go:117] "RemoveContainer" containerID="f0ee788207f6d5632e7216c518742da717d886f9ab394a5bb4eeb278babd44ff"
Jul 20 18:19:28.135016 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:19:28.135001    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0ee788207f6d5632e7216c518742da717d886f9ab394a5bb4eeb278babd44ff\": container with ID starting with f0ee788207f6d5632e7216c518742da717d886f9ab394a5bb4eeb278babd44ff not found: ID does not exist" containerID="f0ee788207f6d5632e7216c518742da717d886f9ab394a5bb4eeb278babd44ff"
Jul 20 18:19:28.135082 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.135019    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0ee788207f6d5632e7216c518742da717d886f9ab394a5bb4eeb278babd44ff"} err="failed to get container status \"f0ee788207f6d5632e7216c518742da717d886f9ab394a5bb4eeb278babd44ff\": rpc error: code = NotFound desc = could not find container \"f0ee788207f6d5632e7216c518742da717d886f9ab394a5bb4eeb278babd44ff\": container with ID starting with f0ee788207f6d5632e7216c518742da717d886f9ab394a5bb4eeb278babd44ff not found: ID does not exist"
Jul 20 18:19:28.135082 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.135031    2569 scope.go:117] "RemoveContainer" containerID="9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc"
Jul 20 18:19:28.135240 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.135225    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc"} err="failed to get container status \"9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc\": rpc error: code = NotFound desc = could not find container \"9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc\": container with ID starting with 9add43c179a73e0e54db5da09ce5aeecc23748c80b4ee1ab8dde721edd43c2fc not found: ID does not exist"
Jul 20 18:19:28.135280 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.135240    2569 scope.go:117] "RemoveContainer" containerID="0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197"
Jul 20 18:19:28.135461 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.135436    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197"} err="failed to get container status \"0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197\": rpc error: code = NotFound desc = could not find container \"0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197\": container with ID starting with 0518a8e3816dd09ce8aa1a1080c12d4616253dda1db232947645c6d354f8a197 not found: ID does not exist"
Jul 20 18:19:28.135461 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.135452    2569 scope.go:117] "RemoveContainer" containerID="9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573"
Jul 20 18:19:28.135650 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.135635    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573"} err="failed to get container status \"9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573\": rpc error: code = NotFound desc = could not find container \"9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573\": container with ID starting with 9eba9e8bb9ed5ea9df08c63bfc7ac559dd001205a1aacf54e0b70d63bb21b573 not found: ID does not exist"
Jul 20 18:19:28.135695 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.135650    2569 scope.go:117] "RemoveContainer" containerID="dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e"
Jul 20 18:19:28.135824 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.135808    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e"} err="failed to get container status \"dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e\": rpc error: code = NotFound desc = could not find container \"dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e\": container with ID starting with dc116dc2242eaf3c5cec34f477cd4fa8b2b0c7818a7402986480e49db404ec2e not found: ID does not exist"
Jul 20 18:19:28.135864 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.135824    2569 scope.go:117] "RemoveContainer" containerID="93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363"
Jul 20 18:19:28.136039 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.136014    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363"} err="failed to get container status \"93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363\": rpc error: code = NotFound desc = could not find container \"93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363\": container with ID starting with 93df6a5c689c638be252d87ff85bd242e94505572722101aefd2523b769ea363 not found: ID does not exist"
Jul 20 18:19:28.136039 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.136034    2569 scope.go:117] "RemoveContainer" containerID="88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071"
Jul 20 18:19:28.136319 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.136303    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071"} err="failed to get container status \"88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071\": rpc error: code = NotFound desc = could not find container \"88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071\": container with ID starting with 88febdb2bbbcd9f744624bdd97c709e020a360711fa9fd169020568ab77c1071 not found: ID does not exist"
Jul 20 18:19:28.136366 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.136320    2569 scope.go:117] "RemoveContainer" containerID="f0ee788207f6d5632e7216c518742da717d886f9ab394a5bb4eeb278babd44ff"
Jul 20 18:19:28.136550 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.136534    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0ee788207f6d5632e7216c518742da717d886f9ab394a5bb4eeb278babd44ff"} err="failed to get container status \"f0ee788207f6d5632e7216c518742da717d886f9ab394a5bb4eeb278babd44ff\": rpc error: code = NotFound desc = could not find container \"f0ee788207f6d5632e7216c518742da717d886f9ab394a5bb4eeb278babd44ff\": container with ID starting with f0ee788207f6d5632e7216c518742da717d886f9ab394a5bb4eeb278babd44ff not found: ID does not exist"
Jul 20 18:19:28.141564 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141540    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 18:19:28.141835 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141824    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="config-reloader"
Jul 20 18:19:28.141876 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141837    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="config-reloader"
Jul 20 18:19:28.141876 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141848    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="prom-label-proxy"
Jul 20 18:19:28.141876 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141854    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="prom-label-proxy"
Jul 20 18:19:28.141876 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141862    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fe2898d-c51c-4fe4-a2c5-c554e9e499be" containerName="registry"
Jul 20 18:19:28.141876 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141867    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fe2898d-c51c-4fe4-a2c5-c554e9e499be" containerName="registry"
Jul 20 18:19:28.141876 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141873    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="kube-rbac-proxy"
Jul 20 18:19:28.141876 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141878    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="kube-rbac-proxy"
Jul 20 18:19:28.142086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141886    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="kube-rbac-proxy-metric"
Jul 20 18:19:28.142086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141891    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="kube-rbac-proxy-metric"
Jul 20 18:19:28.142086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141901    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="kube-rbac-proxy-web"
Jul 20 18:19:28.142086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141906    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="kube-rbac-proxy-web"
Jul 20 18:19:28.142086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141918    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="init-config-reloader"
Jul 20 18:19:28.142086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141923    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="init-config-reloader"
Jul 20 18:19:28.142086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141929    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="alertmanager"
Jul 20 18:19:28.142086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141934    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="alertmanager"
Jul 20 18:19:28.142086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141977    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="kube-rbac-proxy"
Jul 20 18:19:28.142086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141987    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="kube-rbac-proxy-web"
Jul 20 18:19:28.142086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141993    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="config-reloader"
Jul 20 18:19:28.142086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.141998    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="alertmanager"
Jul 20 18:19:28.142086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.142003    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="kube-rbac-proxy-metric"
Jul 20 18:19:28.142086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.142009    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4234579d-ed4a-4e12-8afd-881271427a53" containerName="prom-label-proxy"
Jul 20 18:19:28.142086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.142014    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fe2898d-c51c-4fe4-a2c5-c554e9e499be" containerName="registry"
Jul 20 18:19:28.146300 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.146277    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4234579d-ed4a-4e12-8afd-881271427a53-config-out\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:28.146300 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.146301    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4234579d-ed4a-4e12-8afd-881271427a53-tls-assets\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:28.146413 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.146310    2569 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-config-volume\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:28.146413 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.146319    2569 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-cluster-tls-config\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:28.146413 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.146327    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-main-tls\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:28.146413 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.146337    2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4234579d-ed4a-4e12-8afd-881271427a53-alertmanager-main-db\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:28.146413 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.146346    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:28.146413 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.146354    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-web-config\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:28.146413 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.146362    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:28.146413 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.146370    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgczj\" (UniqueName: \"kubernetes.io/projected/4234579d-ed4a-4e12-8afd-881271427a53-kube-api-access-dgczj\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:28.146413 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.146379    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4234579d-ed4a-4e12-8afd-881271427a53-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:28.146674 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.146591    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.149707 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.149684    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 20 18:19:28.149832 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.149685    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 20 18:19:28.149956 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.149943    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 20 18:19:28.150211 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.150189    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-cbxld\""
Jul 20 18:19:28.150286 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.150221    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 20 18:19:28.150286 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.150251    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 20 18:19:28.150286 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.150203    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 20 18:19:28.150440 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.150351    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 20 18:19:28.150440 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.150357    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 20 18:19:28.156611 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.156585    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 20 18:19:28.163131 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.163108    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 18:19:28.247598 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.247558    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-config-volume\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.247598 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.247599    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.247836 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.247620    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f310bdf9-6211-4281-9e13-40f481c9d5f7-config-out\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.247836 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.247636    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f310bdf9-6211-4281-9e13-40f481c9d5f7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.247836 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.247679    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.247836 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.247709    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f310bdf9-6211-4281-9e13-40f481c9d5f7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.247836 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.247737    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.247836 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.247759    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-web-config\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.247836 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.247773    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrsdt\" (UniqueName: \"kubernetes.io/projected/f310bdf9-6211-4281-9e13-40f481c9d5f7-kube-api-access-lrsdt\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.248148 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.247852    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f310bdf9-6211-4281-9e13-40f481c9d5f7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.248148 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.247900    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.248148 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.247947    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f310bdf9-6211-4281-9e13-40f481c9d5f7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.248148 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.247997    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.348758 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.348656    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.348758 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.348718    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-config-volume\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.348758 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.348743    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.349010 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.348768    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f310bdf9-6211-4281-9e13-40f481c9d5f7-config-out\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.349010 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.348792    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f310bdf9-6211-4281-9e13-40f481c9d5f7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.349010 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.348836    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.349010 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.348877    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f310bdf9-6211-4281-9e13-40f481c9d5f7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.349010 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.348909    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.349010 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.348944    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-web-config\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.349010 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.348968    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrsdt\" (UniqueName: \"kubernetes.io/projected/f310bdf9-6211-4281-9e13-40f481c9d5f7-kube-api-access-lrsdt\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.349010 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.348998    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f310bdf9-6211-4281-9e13-40f481c9d5f7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.349479 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.349021    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.349479 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.349094    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f310bdf9-6211-4281-9e13-40f481c9d5f7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.349805 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.349775    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f310bdf9-6211-4281-9e13-40f481c9d5f7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.349887 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.349817    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f310bdf9-6211-4281-9e13-40f481c9d5f7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.349887 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.349858    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f310bdf9-6211-4281-9e13-40f481c9d5f7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.351716 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.351688    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f310bdf9-6211-4281-9e13-40f481c9d5f7-config-out\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.351994 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.351970    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-config-volume\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.352115 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.352092    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.352115 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.352098    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.352282 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.352259    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f310bdf9-6211-4281-9e13-40f481c9d5f7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.352406 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.352387    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.352497 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.352481    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-web-config\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.352665 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.352645    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.353690 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.353673    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f310bdf9-6211-4281-9e13-40f481c9d5f7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.357163 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.357145    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrsdt\" (UniqueName: \"kubernetes.io/projected/f310bdf9-6211-4281-9e13-40f481c9d5f7-kube-api-access-lrsdt\") pod \"alertmanager-main-0\" (UID: \"f310bdf9-6211-4281-9e13-40f481c9d5f7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.456751 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.456714    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 20 18:19:28.593762 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:28.593738    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 18:19:28.595548 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:19:28.595525    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf310bdf9_6211_4281_9e13_40f481c9d5f7.slice/crio-7a3f4922cd1c30526291172d383095d7a58e9e03bde2fab438223174848cd33f WatchSource:0}: Error finding container 7a3f4922cd1c30526291172d383095d7a58e9e03bde2fab438223174848cd33f: Status 404 returned error can't find the container with id 7a3f4922cd1c30526291172d383095d7a58e9e03bde2fab438223174848cd33f
Jul 20 18:19:29.086048 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:29.086014    2569 generic.go:356] "Generic (PLEG): container finished" podID="f310bdf9-6211-4281-9e13-40f481c9d5f7" containerID="44ecf1abf89defa777aedcb6591cbfe7ae89cd568a8aae711a4bc6de17a80573" exitCode=0
Jul 20 18:19:29.086048 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:29.086067    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f310bdf9-6211-4281-9e13-40f481c9d5f7","Type":"ContainerDied","Data":"44ecf1abf89defa777aedcb6591cbfe7ae89cd568a8aae711a4bc6de17a80573"}
Jul 20 18:19:29.086590 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:29.086116    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f310bdf9-6211-4281-9e13-40f481c9d5f7","Type":"ContainerStarted","Data":"7a3f4922cd1c30526291172d383095d7a58e9e03bde2fab438223174848cd33f"}
Jul 20 18:19:29.762579 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:29.762544    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4234579d-ed4a-4e12-8afd-881271427a53" path="/var/lib/kubelet/pods/4234579d-ed4a-4e12-8afd-881271427a53/volumes"
Jul 20 18:19:30.093042 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.092955    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f310bdf9-6211-4281-9e13-40f481c9d5f7","Type":"ContainerStarted","Data":"f80e9978c699187daa81f332efc00ad94fde37c84fd2cece326a507aa3cb7d28"}
Jul 20 18:19:30.093042 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.093036    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f310bdf9-6211-4281-9e13-40f481c9d5f7","Type":"ContainerStarted","Data":"e227d7118389e09b637e614fb2982708102db96549b973ac6a134e920d957f32"}
Jul 20 18:19:30.093469 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.093097    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f310bdf9-6211-4281-9e13-40f481c9d5f7","Type":"ContainerStarted","Data":"ab17f7552d5aa42dd8b987509efbf5f7e3682c7110f76215c15851687da377be"}
Jul 20 18:19:30.093469 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.093138    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f310bdf9-6211-4281-9e13-40f481c9d5f7","Type":"ContainerStarted","Data":"3d7d71037037aedbda46a4d3b0d0eff3efd9dfeee417b193d0a7fb3f6fa7d67b"}
Jul 20 18:19:30.093469 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.093173    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f310bdf9-6211-4281-9e13-40f481c9d5f7","Type":"ContainerStarted","Data":"4ee7971e3d949746df27766be601156d255f46476ef2d6374fde6ca0bf6ce3e9"}
Jul 20 18:19:30.093469 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.093208    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f310bdf9-6211-4281-9e13-40f481c9d5f7","Type":"ContainerStarted","Data":"69f278b7ca11f7d1ede76ea0e09b36839baea36d5da66350ad9d9e0bdf55b389"}
Jul 20 18:19:30.140507 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.140456    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.125262729 podStartE2EDuration="2.140439278s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.176549ms" isStatefulPod=false podCreationTimestamp="2026-07-20 18:19:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:19:30.139511169 +0000 UTC m=+366.973483557" watchObservedRunningTime="2026-07-20 18:19:30.140439278 +0000 UTC m=+366.974411666"
Jul 20 18:19:30.745591 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.745556    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-564f949944-bc7cx"]
Jul 20 18:19:30.749868 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.749848    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.755894 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.755871    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 20 18:19:30.755894 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.755885    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 20 18:19:30.756102 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.756082    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-p5hr6\""
Jul 20 18:19:30.759342 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.759326    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 20 18:19:30.760268 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.760254    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 20 18:19:30.779621 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.779598    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 20 18:19:30.787399 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.787369    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-564f949944-bc7cx"]
Jul 20 18:19:30.797208 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.797189    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 20 18:19:30.870319 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.870278    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04484408-c34f-4c9e-8efe-48871f99c4f7-serving-certs-ca-bundle\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.870319 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.870317    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04484408-c34f-4c9e-8efe-48871f99c4f7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.870566 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.870406    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/04484408-c34f-4c9e-8efe-48871f99c4f7-telemeter-client-tls\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.870566 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.870471    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/04484408-c34f-4c9e-8efe-48871f99c4f7-federate-client-tls\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.870566 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.870517    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv8bj\" (UniqueName: \"kubernetes.io/projected/04484408-c34f-4c9e-8efe-48871f99c4f7-kube-api-access-kv8bj\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.870696 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.870627    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04484408-c34f-4c9e-8efe-48871f99c4f7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.870696 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.870665    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04484408-c34f-4c9e-8efe-48871f99c4f7-metrics-client-ca\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.870770 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.870719    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/04484408-c34f-4c9e-8efe-48871f99c4f7-secret-telemeter-client\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.971805 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.971769    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04484408-c34f-4c9e-8efe-48871f99c4f7-metrics-client-ca\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.972040 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.971813    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/04484408-c34f-4c9e-8efe-48871f99c4f7-secret-telemeter-client\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.972040 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.971844    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04484408-c34f-4c9e-8efe-48871f99c4f7-serving-certs-ca-bundle\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.972040 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.971866    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04484408-c34f-4c9e-8efe-48871f99c4f7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.972040 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.971905    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/04484408-c34f-4c9e-8efe-48871f99c4f7-telemeter-client-tls\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.972040 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.971952    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/04484408-c34f-4c9e-8efe-48871f99c4f7-federate-client-tls\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.972040 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.971987    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv8bj\" (UniqueName: \"kubernetes.io/projected/04484408-c34f-4c9e-8efe-48871f99c4f7-kube-api-access-kv8bj\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.972040 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.972031    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04484408-c34f-4c9e-8efe-48871f99c4f7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.972646 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.972613    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04484408-c34f-4c9e-8efe-48871f99c4f7-metrics-client-ca\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.972646 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.972642    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04484408-c34f-4c9e-8efe-48871f99c4f7-serving-certs-ca-bundle\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.972989 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.972965    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04484408-c34f-4c9e-8efe-48871f99c4f7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.974489 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.974426    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/04484408-c34f-4c9e-8efe-48871f99c4f7-telemeter-client-tls\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.974591 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.974566    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/04484408-c34f-4c9e-8efe-48871f99c4f7-federate-client-tls\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.974780 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.974763    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04484408-c34f-4c9e-8efe-48871f99c4f7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.974955 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.974932    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/04484408-c34f-4c9e-8efe-48871f99c4f7-secret-telemeter-client\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:30.980390 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:30.980371    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv8bj\" (UniqueName: \"kubernetes.io/projected/04484408-c34f-4c9e-8efe-48871f99c4f7-kube-api-access-kv8bj\") pod \"telemeter-client-564f949944-bc7cx\" (UID: \"04484408-c34f-4c9e-8efe-48871f99c4f7\") " pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:31.059641 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.059535    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-564f949944-bc7cx"
Jul 20 18:19:31.061624 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.061598    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 18:19:31.061891 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.061865    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="prometheus" containerID="cri-o://23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da" gracePeriod=600
Jul 20 18:19:31.061953 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.061889    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="thanos-sidecar" containerID="cri-o://6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47" gracePeriod=600
Jul 20 18:19:31.061953 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.061902    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="config-reloader" containerID="cri-o://06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c" gracePeriod=600
Jul 20 18:19:31.061953 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.061887    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="kube-rbac-proxy" containerID="cri-o://2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa" gracePeriod=600
Jul 20 18:19:31.062134 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.061946    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="kube-rbac-proxy-web" containerID="cri-o://045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38" gracePeriod=600
Jul 20 18:19:31.062134 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.061981    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd" gracePeriod=600
Jul 20 18:19:31.205714 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.205683    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-564f949944-bc7cx"]
Jul 20 18:19:31.209015 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:19:31.208989    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04484408_c34f_4c9e_8efe_48871f99c4f7.slice/crio-c14c5898a8d9c522ed063481c85eadc9b280a6217b43811b8cb92f271954e3c5 WatchSource:0}: Error finding container c14c5898a8d9c522ed063481c85eadc9b280a6217b43811b8cb92f271954e3c5: Status 404 returned error can't find the container with id c14c5898a8d9c522ed063481c85eadc9b280a6217b43811b8cb92f271954e3c5
Jul 20 18:19:31.211617 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.211601    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 18:19:31.313448 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.313383    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:31.476881 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.476838    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.476881 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.476887    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-metrics-client-certs\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477175 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.476931    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-kubelet-serving-ca-bundle\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477175 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.476957    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-serving-certs-ca-bundle\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477175 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.476986    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-db\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477175 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477018    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-tls\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477175 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477045    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-config\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477175 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477094    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhpl2\" (UniqueName: \"kubernetes.io/projected/e5ccaaca-901d-476e-aeab-74cd33dbf59a-kube-api-access-hhpl2\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477175 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477116    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-grpc-tls\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477175 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477160    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477588 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477217    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-metrics-client-ca\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477588 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477262    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-thanos-prometheus-http-client-file\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477588 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477293    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-trusted-ca-bundle\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477588 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477320    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e5ccaaca-901d-476e-aeab-74cd33dbf59a-tls-assets\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477588 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477350    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-web-config\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477588 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477325    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:19:31.477588 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477478    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-1\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477588 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477519    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-2\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477588 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477551    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-kube-rbac-proxy\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.477588 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477576    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-0\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.478109 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477604    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e5ccaaca-901d-476e-aeab-74cd33dbf59a-config-out\") pod \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\" (UID: \"e5ccaaca-901d-476e-aeab-74cd33dbf59a\") "
Jul 20 18:19:31.478109 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.477944    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.478754 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.478469    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 18:19:31.478754 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.478582    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:19:31.479813 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.479551    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:31.479813 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.479663    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:31.480025 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.479866    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:31.480565 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.480162    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:19:31.480565 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.480182    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5ccaaca-901d-476e-aeab-74cd33dbf59a-config-out" (OuterVolumeSpecName: "config-out") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 18:19:31.480565 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.480290    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:19:31.480565 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.480529    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:19:31.480824 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.480775    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:31.480885 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.480848    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:31.481233 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.481194    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:19:31.481373 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.481343    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5ccaaca-901d-476e-aeab-74cd33dbf59a-kube-api-access-hhpl2" (OuterVolumeSpecName: "kube-api-access-hhpl2") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "kube-api-access-hhpl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:19:31.481939 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.481914    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:31.482084 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.482041    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:19:31.482216 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.482195    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:31.482785 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.482767    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5ccaaca-901d-476e-aeab-74cd33dbf59a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:19:31.483158 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.483140    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-config" (OuterVolumeSpecName: "config") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:31.490984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.490966    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-web-config" (OuterVolumeSpecName: "web-config") pod "e5ccaaca-901d-476e-aeab-74cd33dbf59a" (UID: "e5ccaaca-901d-476e-aeab-74cd33dbf59a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:19:31.578996 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.578915    2569 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-metrics-client-certs\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.578996 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.578942    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.578996 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.578953    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-db\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.578996 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.578961    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.578996 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.578970    2569 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-config\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.578996 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.578978    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhpl2\" (UniqueName: \"kubernetes.io/projected/e5ccaaca-901d-476e-aeab-74cd33dbf59a-kube-api-access-hhpl2\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.578996 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.578986    2569 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-grpc-tls\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.578996 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.578995    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.578996 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.579003    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-configmap-metrics-client-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.579402 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.579012    2569 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.579402 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.579022    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.579402 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.579030    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e5ccaaca-901d-476e-aeab-74cd33dbf59a-tls-assets\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.579402 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.579037    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-web-config\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.579402 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.579044    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.579402 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.579072    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.579402 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.579081    2569 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-kube-rbac-proxy\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.579402 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.579089    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e5ccaaca-901d-476e-aeab-74cd33dbf59a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.579402 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.579096    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e5ccaaca-901d-476e-aeab-74cd33dbf59a-config-out\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:31.579402 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:31.579106    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e5ccaaca-901d-476e-aeab-74cd33dbf59a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:19:32.105225 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.105189    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-564f949944-bc7cx" event={"ID":"04484408-c34f-4c9e-8efe-48871f99c4f7","Type":"ContainerStarted","Data":"c14c5898a8d9c522ed063481c85eadc9b280a6217b43811b8cb92f271954e3c5"}
Jul 20 18:19:32.108358 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.108330    2569 generic.go:356] "Generic (PLEG): container finished" podID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerID="fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd" exitCode=0
Jul 20 18:19:32.108358 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.108356    2569 generic.go:356] "Generic (PLEG): container finished" podID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerID="2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa" exitCode=0
Jul 20 18:19:32.108548 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.108369    2569 generic.go:356] "Generic (PLEG): container finished" podID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerID="045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38" exitCode=0
Jul 20 18:19:32.108548 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.108385    2569 generic.go:356] "Generic (PLEG): container finished" podID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerID="6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47" exitCode=0
Jul 20 18:19:32.108548 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.108395    2569 generic.go:356] "Generic (PLEG): container finished" podID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerID="06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c" exitCode=0
Jul 20 18:19:32.108548 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.108404    2569 generic.go:356] "Generic (PLEG): container finished" podID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerID="23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da" exitCode=0
Jul 20 18:19:32.108548 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.108408    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.108548 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.108364    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e5ccaaca-901d-476e-aeab-74cd33dbf59a","Type":"ContainerDied","Data":"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd"}
Jul 20 18:19:32.108548 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.108510    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e5ccaaca-901d-476e-aeab-74cd33dbf59a","Type":"ContainerDied","Data":"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa"}
Jul 20 18:19:32.108898 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.108569    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e5ccaaca-901d-476e-aeab-74cd33dbf59a","Type":"ContainerDied","Data":"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38"}
Jul 20 18:19:32.108898 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.108632    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e5ccaaca-901d-476e-aeab-74cd33dbf59a","Type":"ContainerDied","Data":"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47"}
Jul 20 18:19:32.108898 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.108571    2569 scope.go:117] "RemoveContainer" containerID="fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd"
Jul 20 18:19:32.108898 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.108686    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e5ccaaca-901d-476e-aeab-74cd33dbf59a","Type":"ContainerDied","Data":"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c"}
Jul 20 18:19:32.108898 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.108735    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e5ccaaca-901d-476e-aeab-74cd33dbf59a","Type":"ContainerDied","Data":"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da"}
Jul 20 18:19:32.108898 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.108787    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e5ccaaca-901d-476e-aeab-74cd33dbf59a","Type":"ContainerDied","Data":"aa16cc170f61a3c20b1f6f123d31c7d5f9e1e5fffd2d5c4961e8f5f8e684a084"}
Jul 20 18:19:32.116506 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.116488    2569 scope.go:117] "RemoveContainer" containerID="2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa"
Jul 20 18:19:32.124170 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.124154    2569 scope.go:117] "RemoveContainer" containerID="045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38"
Jul 20 18:19:32.130906 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.130887    2569 scope.go:117] "RemoveContainer" containerID="6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47"
Jul 20 18:19:32.132725 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.132704    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 18:19:32.137603 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.137578    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 18:19:32.140617 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.140548    2569 scope.go:117] "RemoveContainer" containerID="06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c"
Jul 20 18:19:32.148016 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.147999    2569 scope.go:117] "RemoveContainer" containerID="23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da"
Jul 20 18:19:32.155443 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.155420    2569 scope.go:117] "RemoveContainer" containerID="a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b"
Jul 20 18:19:32.162508 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.162485    2569 scope.go:117] "RemoveContainer" containerID="fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd"
Jul 20 18:19:32.162765 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:19:32.162744    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd\": container with ID starting with fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd not found: ID does not exist" containerID="fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd"
Jul 20 18:19:32.162819 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.162774    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd"} err="failed to get container status \"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd\": rpc error: code = NotFound desc = could not find container \"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd\": container with ID starting with fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd not found: ID does not exist"
Jul 20 18:19:32.162819 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.162794    2569 scope.go:117] "RemoveContainer" containerID="2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa"
Jul 20 18:19:32.162996 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:19:32.162977    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa\": container with ID starting with 2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa not found: ID does not exist" containerID="2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa"
Jul 20 18:19:32.163037 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.163004    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa"} err="failed to get container status \"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa\": rpc error: code = NotFound desc = could not find container \"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa\": container with ID starting with 2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa not found: ID does not exist"
Jul 20 18:19:32.163037 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.163031    2569 scope.go:117] "RemoveContainer" containerID="045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38"
Jul 20 18:19:32.163286 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:19:32.163239    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38\": container with ID starting with 045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38 not found: ID does not exist" containerID="045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38"
Jul 20 18:19:32.163286 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.163263    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38"} err="failed to get container status \"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38\": rpc error: code = NotFound desc = could not find container \"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38\": container with ID starting with 045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38 not found: ID does not exist"
Jul 20 18:19:32.163286 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.163281    2569 scope.go:117] "RemoveContainer" containerID="6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47"
Jul 20 18:19:32.163550 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:19:32.163534    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47\": container with ID starting with 6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47 not found: ID does not exist" containerID="6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47"
Jul 20 18:19:32.163609 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.163557    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47"} err="failed to get container status \"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47\": rpc error: code = NotFound desc = could not find container \"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47\": container with ID starting with 6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47 not found: ID does not exist"
Jul 20 18:19:32.163609 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.163576    2569 scope.go:117] "RemoveContainer" containerID="06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c"
Jul 20 18:19:32.163878 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:19:32.163854    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c\": container with ID starting with 06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c not found: ID does not exist" containerID="06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c"
Jul 20 18:19:32.163932 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.163885    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c"} err="failed to get container status \"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c\": rpc error: code = NotFound desc = could not find container \"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c\": container with ID starting with 06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c not found: ID does not exist"
Jul 20 18:19:32.163932 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.163904    2569 scope.go:117] "RemoveContainer" containerID="23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da"
Jul 20 18:19:32.164256 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:19:32.164231    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da\": container with ID starting with 23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da not found: ID does not exist" containerID="23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da"
Jul 20 18:19:32.164375 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.164260    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da"} err="failed to get container status \"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da\": rpc error: code = NotFound desc = could not find container \"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da\": container with ID starting with 23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da not found: ID does not exist"
Jul 20 18:19:32.164375 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.164278    2569 scope.go:117] "RemoveContainer" containerID="a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b"
Jul 20 18:19:32.164608 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:19:32.164588    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b\": container with ID starting with a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b not found: ID does not exist" containerID="a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b"
Jul 20 18:19:32.164659 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.164609    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b"} err="failed to get container status \"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b\": rpc error: code = NotFound desc = could not find container \"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b\": container with ID starting with a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b not found: ID does not exist"
Jul 20 18:19:32.164659 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.164624    2569 scope.go:117] "RemoveContainer" containerID="fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd"
Jul 20 18:19:32.164884 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.164862    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd"} err="failed to get container status \"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd\": rpc error: code = NotFound desc = could not find container \"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd\": container with ID starting with fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd not found: ID does not exist"
Jul 20 18:19:32.164943 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.164887    2569 scope.go:117] "RemoveContainer" containerID="2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa"
Jul 20 18:19:32.165147 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165128    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa"} err="failed to get container status \"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa\": rpc error: code = NotFound desc = could not find container \"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa\": container with ID starting with 2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa not found: ID does not exist"
Jul 20 18:19:32.165187 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165148    2569 scope.go:117] "RemoveContainer" containerID="045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38"
Jul 20 18:19:32.165383 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165363    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38"} err="failed to get container status \"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38\": rpc error: code = NotFound desc = could not find container \"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38\": container with ID starting with 045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38 not found: ID does not exist"
Jul 20 18:19:32.165439 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165384    2569 scope.go:117] "RemoveContainer" containerID="6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47"
Jul 20 18:19:32.165526 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165511    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 18:19:32.165601 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165585    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47"} err="failed to get container status \"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47\": rpc error: code = NotFound desc = could not find container \"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47\": container with ID starting with 6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47 not found: ID does not exist"
Jul 20 18:19:32.165650 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165604    2569 scope.go:117] "RemoveContainer" containerID="06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c"
Jul 20 18:19:32.165814 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165793    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c"} err="failed to get container status \"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c\": rpc error: code = NotFound desc = could not find container \"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c\": container with ID starting with 06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c not found: ID does not exist"
Jul 20 18:19:32.165860 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165816    2569 scope.go:117] "RemoveContainer" containerID="23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da"
Jul 20 18:19:32.165896 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165870    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="kube-rbac-proxy"
Jul 20 18:19:32.165896 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165884    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="kube-rbac-proxy"
Jul 20 18:19:32.165896 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165896    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="init-config-reloader"
Jul 20 18:19:32.166009 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165903    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="init-config-reloader"
Jul 20 18:19:32.166009 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165910    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="thanos-sidecar"
Jul 20 18:19:32.166009 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165916    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="thanos-sidecar"
Jul 20 18:19:32.166009 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165924    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="config-reloader"
Jul 20 18:19:32.166009 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165929    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="config-reloader"
Jul 20 18:19:32.166009 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165934    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="kube-rbac-proxy-web"
Jul 20 18:19:32.166009 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165939    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="kube-rbac-proxy-web"
Jul 20 18:19:32.166009 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165957    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="kube-rbac-proxy-thanos"
Jul 20 18:19:32.166009 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165964    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="kube-rbac-proxy-thanos"
Jul 20 18:19:32.166009 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165981    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="prometheus"
Jul 20 18:19:32.166009 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.165989    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="prometheus"
Jul 20 18:19:32.166009 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.166005    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da"} err="failed to get container status \"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da\": rpc error: code = NotFound desc = could not find container \"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da\": container with ID starting with 23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da not found: ID does not exist"
Jul 20 18:19:32.166485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.166022    2569 scope.go:117] "RemoveContainer" containerID="a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b"
Jul 20 18:19:32.166485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.166075    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="kube-rbac-proxy"
Jul 20 18:19:32.166485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.166087    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="prometheus"
Jul 20 18:19:32.166485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.166100    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="kube-rbac-proxy-thanos"
Jul 20 18:19:32.166485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.166109    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="config-reloader"
Jul 20 18:19:32.166485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.166120    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="kube-rbac-proxy-web"
Jul 20 18:19:32.166485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.166127    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" containerName="thanos-sidecar"
Jul 20 18:19:32.166485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.166280    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b"} err="failed to get container status \"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b\": rpc error: code = NotFound desc = could not find container \"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b\": container with ID starting with a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b not found: ID does not exist"
Jul 20 18:19:32.166485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.166296    2569 scope.go:117] "RemoveContainer" containerID="fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd"
Jul 20 18:19:32.166485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.166471    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd"} err="failed to get container status \"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd\": rpc error: code = NotFound desc = could not find container \"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd\": container with ID starting with fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd not found: ID does not exist"
Jul 20 18:19:32.166485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.166487    2569 scope.go:117] "RemoveContainer" containerID="2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa"
Jul 20 18:19:32.166894 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.166691    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa"} err="failed to get container status \"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa\": rpc error: code = NotFound desc = could not find container \"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa\": container with ID starting with 2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa not found: ID does not exist"
Jul 20 18:19:32.166894 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.166707    2569 scope.go:117] "RemoveContainer" containerID="045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38"
Jul 20 18:19:32.166960 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.166910    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38"} err="failed to get container status \"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38\": rpc error: code = NotFound desc = could not find container \"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38\": container with ID starting with 045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38 not found: ID does not exist"
Jul 20 18:19:32.166960 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.166930    2569 scope.go:117] "RemoveContainer" containerID="6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47"
Jul 20 18:19:32.167169 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.167150    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47"} err="failed to get container status \"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47\": rpc error: code = NotFound desc = could not find container \"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47\": container with ID starting with 6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47 not found: ID does not exist"
Jul 20 18:19:32.167221 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.167169    2569 scope.go:117] "RemoveContainer" containerID="06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c"
Jul 20 18:19:32.167374 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.167352    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c"} err="failed to get container status \"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c\": rpc error: code = NotFound desc = could not find container \"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c\": container with ID starting with 06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c not found: ID does not exist"
Jul 20 18:19:32.167418 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.167375    2569 scope.go:117] "RemoveContainer" containerID="23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da"
Jul 20 18:19:32.167557 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.167536    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da"} err="failed to get container status \"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da\": rpc error: code = NotFound desc = could not find container \"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da\": container with ID starting with 23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da not found: ID does not exist"
Jul 20 18:19:32.167557 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.167556    2569 scope.go:117] "RemoveContainer" containerID="a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b"
Jul 20 18:19:32.167762 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.167733    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b"} err="failed to get container status \"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b\": rpc error: code = NotFound desc = could not find container \"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b\": container with ID starting with a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b not found: ID does not exist"
Jul 20 18:19:32.167833 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.167764    2569 scope.go:117] "RemoveContainer" containerID="fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd"
Jul 20 18:19:32.168011 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.167995    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd"} err="failed to get container status \"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd\": rpc error: code = NotFound desc = could not find container \"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd\": container with ID starting with fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd not found: ID does not exist"
Jul 20 18:19:32.168066 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.168012    2569 scope.go:117] "RemoveContainer" containerID="2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa"
Jul 20 18:19:32.168248 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.168232    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa"} err="failed to get container status \"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa\": rpc error: code = NotFound desc = could not find container \"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa\": container with ID starting with 2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa not found: ID does not exist"
Jul 20 18:19:32.168297 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.168248    2569 scope.go:117] "RemoveContainer" containerID="045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38"
Jul 20 18:19:32.168436 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.168421    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38"} err="failed to get container status \"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38\": rpc error: code = NotFound desc = could not find container \"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38\": container with ID starting with 045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38 not found: ID does not exist"
Jul 20 18:19:32.168473 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.168436    2569 scope.go:117] "RemoveContainer" containerID="6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47"
Jul 20 18:19:32.168636 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.168620    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47"} err="failed to get container status \"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47\": rpc error: code = NotFound desc = could not find container \"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47\": container with ID starting with 6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47 not found: ID does not exist"
Jul 20 18:19:32.168699 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.168637    2569 scope.go:117] "RemoveContainer" containerID="06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c"
Jul 20 18:19:32.168858 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.168839    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c"} err="failed to get container status \"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c\": rpc error: code = NotFound desc = could not find container \"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c\": container with ID starting with 06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c not found: ID does not exist"
Jul 20 18:19:32.168907 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.168858    2569 scope.go:117] "RemoveContainer" containerID="23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da"
Jul 20 18:19:32.169097 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.169041    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da"} err="failed to get container status \"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da\": rpc error: code = NotFound desc = could not find container \"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da\": container with ID starting with 23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da not found: ID does not exist"
Jul 20 18:19:32.169097 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.169079    2569 scope.go:117] "RemoveContainer" containerID="a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b"
Jul 20 18:19:32.169313 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.169295    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b"} err="failed to get container status \"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b\": rpc error: code = NotFound desc = could not find container \"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b\": container with ID starting with a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b not found: ID does not exist"
Jul 20 18:19:32.169375 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.169315    2569 scope.go:117] "RemoveContainer" containerID="fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd"
Jul 20 18:19:32.169499 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.169477    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd"} err="failed to get container status \"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd\": rpc error: code = NotFound desc = could not find container \"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd\": container with ID starting with fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd not found: ID does not exist"
Jul 20 18:19:32.169541 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.169500    2569 scope.go:117] "RemoveContainer" containerID="2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa"
Jul 20 18:19:32.169707 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.169690    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa"} err="failed to get container status \"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa\": rpc error: code = NotFound desc = could not find container \"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa\": container with ID starting with 2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa not found: ID does not exist"
Jul 20 18:19:32.169766 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.169709    2569 scope.go:117] "RemoveContainer" containerID="045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38"
Jul 20 18:19:32.169945 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.169927    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38"} err="failed to get container status \"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38\": rpc error: code = NotFound desc = could not find container \"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38\": container with ID starting with 045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38 not found: ID does not exist"
Jul 20 18:19:32.169995 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.169946    2569 scope.go:117] "RemoveContainer" containerID="6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47"
Jul 20 18:19:32.170242 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.170220    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47"} err="failed to get container status \"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47\": rpc error: code = NotFound desc = could not find container \"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47\": container with ID starting with 6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47 not found: ID does not exist"
Jul 20 18:19:32.170317 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.170246    2569 scope.go:117] "RemoveContainer" containerID="06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c"
Jul 20 18:19:32.170474 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.170457    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c"} err="failed to get container status \"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c\": rpc error: code = NotFound desc = could not find container \"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c\": container with ID starting with 06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c not found: ID does not exist"
Jul 20 18:19:32.170521 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.170474    2569 scope.go:117] "RemoveContainer" containerID="23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da"
Jul 20 18:19:32.170659 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.170640    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da"} err="failed to get container status \"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da\": rpc error: code = NotFound desc = could not find container \"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da\": container with ID starting with 23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da not found: ID does not exist"
Jul 20 18:19:32.170707 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.170659    2569 scope.go:117] "RemoveContainer" containerID="a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b"
Jul 20 18:19:32.170855 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.170837    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b"} err="failed to get container status \"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b\": rpc error: code = NotFound desc = could not find container \"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b\": container with ID starting with a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b not found: ID does not exist"
Jul 20 18:19:32.170897 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.170859    2569 scope.go:117] "RemoveContainer" containerID="fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd"
Jul 20 18:19:32.171083 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.171066    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd"} err="failed to get container status \"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd\": rpc error: code = NotFound desc = could not find container \"fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd\": container with ID starting with fe9c8d53de1b877be92fdf13c4c79e65504ac34bd3f653440cc67173ceb763cd not found: ID does not exist"
Jul 20 18:19:32.171136 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.171083    2569 scope.go:117] "RemoveContainer" containerID="2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa"
Jul 20 18:19:32.171324 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.171303    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa"} err="failed to get container status \"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa\": rpc error: code = NotFound desc = could not find container \"2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa\": container with ID starting with 2fc9f8b3a85a75dea6b5ba14fa8d35b751b9e4787dd9640b600b14e5b36372fa not found: ID does not exist"
Jul 20 18:19:32.171324 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.171324    2569 scope.go:117] "RemoveContainer" containerID="045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38"
Jul 20 18:19:32.171540 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.171523    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38"} err="failed to get container status \"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38\": rpc error: code = NotFound desc = could not find container \"045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38\": container with ID starting with 045c68e73b81c42432aea8d02b67b0be4707ead3060f58c3148338c4977d1f38 not found: ID does not exist"
Jul 20 18:19:32.171602 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.171543    2569 scope.go:117] "RemoveContainer" containerID="6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47"
Jul 20 18:19:32.171776 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.171755    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47"} err="failed to get container status \"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47\": rpc error: code = NotFound desc = could not find container \"6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47\": container with ID starting with 6f5be827da76b831a9ae3e5860e38504242cf9aee7a8c1e01734fc570918df47 not found: ID does not exist"
Jul 20 18:19:32.171821 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.171777    2569 scope.go:117] "RemoveContainer" containerID="06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c"
Jul 20 18:19:32.171990 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.171974    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c"} err="failed to get container status \"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c\": rpc error: code = NotFound desc = could not find container \"06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c\": container with ID starting with 06ad6a7240959eb414edcfe75b5429ad1a445745220393ac3390bfd32faaad2c not found: ID does not exist"
Jul 20 18:19:32.172073 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.171992    2569 scope.go:117] "RemoveContainer" containerID="23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da"
Jul 20 18:19:32.172248 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.172233    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.172341 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.172232    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da"} err="failed to get container status \"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da\": rpc error: code = NotFound desc = could not find container \"23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da\": container with ID starting with 23f0440fc4fdd870890722656adeedbfd8ffbd8139b50e8dec2932b24b63c5da not found: ID does not exist"
Jul 20 18:19:32.172411 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.172344    2569 scope.go:117] "RemoveContainer" containerID="a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b"
Jul 20 18:19:32.172810 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.172784    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b"} err="failed to get container status \"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b\": rpc error: code = NotFound desc = could not find container \"a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b\": container with ID starting with a6287e2ac34f8f58ba039fa61fced7d83132562592ac21c2660a8ad5abdfea9b not found: ID does not exist"
Jul 20 18:19:32.174457 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.174439    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 20 18:19:32.174548 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.174463    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 20 18:19:32.174720 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.174704    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-eacu4nhuq66qf\""
Jul 20 18:19:32.175267 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.175251    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 20 18:19:32.175624 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.175608    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 20 18:19:32.175682 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.175610    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 20 18:19:32.178149 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.177915    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 20 18:19:32.178149 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.178009    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 20 18:19:32.178298 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.178204    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 20 18:19:32.178298 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.178283    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 20 18:19:32.178473 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.178458    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 20 18:19:32.178566 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.178504    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 20 18:19:32.178566 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.178509    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 20 18:19:32.178864 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.178846    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xjfsx\""
Jul 20 18:19:32.178864 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.178857    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 20 18:19:32.180680 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.180659    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 20 18:19:32.182618 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.182598    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 20 18:19:32.188030 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.188010    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 18:19:32.284833 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.284789    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/02e37e7f-933e-49bf-9935-90f64fdf43a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.284833 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.284834    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.284859    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.284884    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.284911    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.284926    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.284948    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.284969    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.284994    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/02e37e7f-933e-49bf-9935-90f64fdf43a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.285015    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/02e37e7f-933e-49bf-9935-90f64fdf43a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.285086    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.285128    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hwcd\" (UniqueName: \"kubernetes.io/projected/02e37e7f-933e-49bf-9935-90f64fdf43a1-kube-api-access-7hwcd\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.285171    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.285205    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.285264    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-config\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.285293    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.285341    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.285379    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285932 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.285408    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.285932 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.285439    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386209 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386178    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/02e37e7f-933e-49bf-9935-90f64fdf43a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386209 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386211    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386368 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386229    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386368 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386246    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386368 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386288    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386534 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386410    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386534 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386449    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386534 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386481    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386534 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386524    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/02e37e7f-933e-49bf-9935-90f64fdf43a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386710 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386560    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/02e37e7f-933e-49bf-9935-90f64fdf43a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386710 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386584    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386710 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386605    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hwcd\" (UniqueName: \"kubernetes.io/projected/02e37e7f-933e-49bf-9935-90f64fdf43a1-kube-api-access-7hwcd\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386710 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386633    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386710 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386664    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386960 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386708    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-config\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386960 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386737    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386960 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386775    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386960 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386808    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386960 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386839    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.386960 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386872    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.387294 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.387133    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.387405 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.387385    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.387471 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.387453    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.387531 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.386666    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/02e37e7f-933e-49bf-9935-90f64fdf43a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.387724 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.387696    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.387797 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.387705    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.389150 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.389122    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.389765 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.389721    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.390072 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.390014    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/02e37e7f-933e-49bf-9935-90f64fdf43a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.390167 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.390151    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.390813 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.390610    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.390813 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.390765    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/02e37e7f-933e-49bf-9935-90f64fdf43a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.390965 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.390859    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.391152 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.391130    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.391664 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.391641    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/02e37e7f-933e-49bf-9935-90f64fdf43a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.392090 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.392072    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-config\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.392755 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.392736    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.392876 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.392852    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.393429 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.393413    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/02e37e7f-933e-49bf-9935-90f64fdf43a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.396429 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.396407    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hwcd\" (UniqueName: \"kubernetes.io/projected/02e37e7f-933e-49bf-9935-90f64fdf43a1-kube-api-access-7hwcd\") pod \"prometheus-k8s-0\" (UID: \"02e37e7f-933e-49bf-9935-90f64fdf43a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.482619 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.482579    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:32.630962 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:32.630933    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 18:19:32.633869 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:19:32.633846    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02e37e7f_933e_49bf_9935_90f64fdf43a1.slice/crio-eaa3f3ae18b86b149d4f833cdfd54be31299056a38423e5b1ae4d33de447292d WatchSource:0}: Error finding container eaa3f3ae18b86b149d4f833cdfd54be31299056a38423e5b1ae4d33de447292d: Status 404 returned error can't find the container with id eaa3f3ae18b86b149d4f833cdfd54be31299056a38423e5b1ae4d33de447292d
Jul 20 18:19:33.115584 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:33.115556    2569 generic.go:356] "Generic (PLEG): container finished" podID="02e37e7f-933e-49bf-9935-90f64fdf43a1" containerID="d7fc87b4df778fd79bd2cdd475eaaafab7a3b7a1c76779b0760bd59dd7ea671f" exitCode=0
Jul 20 18:19:33.115758 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:33.115643    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"02e37e7f-933e-49bf-9935-90f64fdf43a1","Type":"ContainerDied","Data":"d7fc87b4df778fd79bd2cdd475eaaafab7a3b7a1c76779b0760bd59dd7ea671f"}
Jul 20 18:19:33.115758 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:33.115707    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"02e37e7f-933e-49bf-9935-90f64fdf43a1","Type":"ContainerStarted","Data":"eaa3f3ae18b86b149d4f833cdfd54be31299056a38423e5b1ae4d33de447292d"}
Jul 20 18:19:33.758652 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:33.758619    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5ccaaca-901d-476e-aeab-74cd33dbf59a" path="/var/lib/kubelet/pods/e5ccaaca-901d-476e-aeab-74cd33dbf59a/volumes"
Jul 20 18:19:34.123036 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:34.122947    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"02e37e7f-933e-49bf-9935-90f64fdf43a1","Type":"ContainerStarted","Data":"f4e687a0655a1be1e1ab1db7bf49bd517ce0ba968a2215101996e772b60f3561"}
Jul 20 18:19:34.123036 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:34.123010    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"02e37e7f-933e-49bf-9935-90f64fdf43a1","Type":"ContainerStarted","Data":"f5ea42e3c5a5153cef231b9466d71759db1034ee3ced7687f88cba835e7e6b80"}
Jul 20 18:19:34.123036 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:34.123035    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"02e37e7f-933e-49bf-9935-90f64fdf43a1","Type":"ContainerStarted","Data":"c958029c6923ca2d7ddbdb3afd532aeac3ff86a27134bb599d1350130b2175c4"}
Jul 20 18:19:34.123306 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:34.123084    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"02e37e7f-933e-49bf-9935-90f64fdf43a1","Type":"ContainerStarted","Data":"358827ea149a4b5911a788d42cf17b066d1f65e9097e79d1328cb2889087ec0c"}
Jul 20 18:19:34.123306 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:34.123113    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"02e37e7f-933e-49bf-9935-90f64fdf43a1","Type":"ContainerStarted","Data":"6e3257788ea358b9571072785472f11904d080fa66e797ac816210b7a0ec6330"}
Jul 20 18:19:34.123306 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:34.123151    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"02e37e7f-933e-49bf-9935-90f64fdf43a1","Type":"ContainerStarted","Data":"4527f124ce7138bdf5b32a49ff1603a339ec0f764dca046b615f4e4b1aa76b52"}
Jul 20 18:19:34.150651 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:34.150567    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.13292678 podStartE2EDuration="2.150524431s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.597651ms" isStatefulPod=false podCreationTimestamp="2026-07-20 18:19:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:19:34.14949594 +0000 UTC m=+370.983468328" watchObservedRunningTime="2026-07-20 18:19:34.150524431 +0000 UTC m=+370.984496818"
Jul 20 18:19:35.135926 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:35.135895    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-564f949944-bc7cx" event={"ID":"04484408-c34f-4c9e-8efe-48871f99c4f7","Type":"ContainerStarted","Data":"5b77071e54e6649329d164344e275afc8839c2a7a6f2e59ae07a56925c18b696"}
Jul 20 18:19:36.142400 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:36.142365    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-564f949944-bc7cx" event={"ID":"04484408-c34f-4c9e-8efe-48871f99c4f7","Type":"ContainerStarted","Data":"f06fa36b547ebb894010e91d22f62fc71e8332b8b3812575a775e9e2e272ac89"}
Jul 20 18:19:36.142871 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:36.142422    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-564f949944-bc7cx" event={"ID":"04484408-c34f-4c9e-8efe-48871f99c4f7","Type":"ContainerStarted","Data":"f1d5720ac910307f6cedbcce0c738623eeb193c7a9b8bc9cb9a9fc1c64e35977"}
Jul 20 18:19:36.166738 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:36.166620    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-564f949944-bc7cx" podStartSLOduration=2.3486392609999998 podStartE2EDuration="6.16660398s" totalImagesPullingTime="3.817964719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:19:36.165979553 +0000 UTC m=+372.999951956" watchObservedRunningTime="2026-07-20 18:19:36.16660398 +0000 UTC m=+373.000576368"
Jul 20 18:19:37.013967 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:37.013936    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:37.113142 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:37.113110    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:37.170090 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:37.170042    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:37.483300 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:37.483201    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:19:37.636918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:37.636874    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs\") pod \"network-metrics-daemon-g4z7w\" (UID: \"1bae9e54-a7d1-46ba-9bb6-9602c8d53962\") " pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:19:37.639908 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:37.639880    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bae9e54-a7d1-46ba-9bb6-9602c8d53962-metrics-certs\") pod \"network-metrics-daemon-g4z7w\" (UID: \"1bae9e54-a7d1-46ba-9bb6-9602c8d53962\") " pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:19:37.688204 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:37.688171    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:37.790622 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:37.790594    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:37.856440 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:37.856414    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n7qbr\""
Jul 20 18:19:37.864286 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:37.864270    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g4z7w"
Jul 20 18:19:37.987929 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:37.987896    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g4z7w"]
Jul 20 18:19:37.990574 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:19:37.990545    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bae9e54_a7d1_46ba_9bb6_9602c8d53962.slice/crio-17622b6728543b6ae7bffcba4b094b1e4256f599fdf29975792785e4e0c71282 WatchSource:0}: Error finding container 17622b6728543b6ae7bffcba4b094b1e4256f599fdf29975792785e4e0c71282: Status 404 returned error can't find the container with id 17622b6728543b6ae7bffcba4b094b1e4256f599fdf29975792785e4e0c71282
Jul 20 18:19:38.152453 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:38.152369    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g4z7w" event={"ID":"1bae9e54-a7d1-46ba-9bb6-9602c8d53962","Type":"ContainerStarted","Data":"17622b6728543b6ae7bffcba4b094b1e4256f599fdf29975792785e4e0c71282"}
Jul 20 18:19:38.823229 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:38.823189    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:38.889330 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:38.889291    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:39.160829 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:39.160788    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g4z7w" event={"ID":"1bae9e54-a7d1-46ba-9bb6-9602c8d53962","Type":"ContainerStarted","Data":"00adb75279181c832586bf599315bc2d30232a74b629f24df2d553456309b33c"}
Jul 20 18:19:39.160999 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:39.160876    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g4z7w" event={"ID":"1bae9e54-a7d1-46ba-9bb6-9602c8d53962","Type":"ContainerStarted","Data":"d20f2c1d84b95cc7c45230f6b7bd2386558f703458480b08410d4e555b9e8714"}
Jul 20 18:19:39.180685 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:39.180638    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-g4z7w" podStartSLOduration=374.239625107 podStartE2EDuration="6m15.180622616s" totalImagesPullingTime="940.997509ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:19:39.178484785 +0000 UTC m=+376.012457173" watchObservedRunningTime="2026-07-20 18:19:39.180622616 +0000 UTC m=+376.014595003"
Jul 20 18:19:39.274617 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:39.274586    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:39.334168 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:39.334093    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:41.732830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:41.732795    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:41.832318 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:41.832287    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:41.887829 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:41.887802    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:47.911905 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:47.911871    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:47.974365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:47.974332    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:19:48.031858 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:19:48.031828    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:20:32.483395 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:20:32.483350    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:20:32.498908 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:20:32.498882    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:20:33.391921 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:20:33.391890    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 18:20:40.941470 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:20:40.941437    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:20:41.041875 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:20:41.041842    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:20:43.626139 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:20:43.626101    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:20:43.684700 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:20:43.684669    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:20:52.907765 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:20:52.907729    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:20:52.962997 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:20:52.962967    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:20:54.021723 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:20:54.021689    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:20:54.121812 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:20:54.121777    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:20:54.176305 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:20:54.176274    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:20:56.754208 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:20:56.754176    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:20:56.853733 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:20:56.853702    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:21:52.423208 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:21:52.423162    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:21:52.519846 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:21:52.519799    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:19.136301 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:19.136227    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:19.990246 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:19.990194    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:22.245485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:22.245451    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:22.349708 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:22.349677    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:23.140442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:23.140405    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:23.241971 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:23.241940    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:24.137006 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:24.136968    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:24.246613 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:24.246583    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:24.772619 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:24.772589    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:24.978825 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:24.978791    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:26.344944 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:26.344913    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:26.764010 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:26.763981    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:29.544991 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:29.544958    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:29.603753 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:29.603725    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:32.138098 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:32.138048    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:32.240981 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:32.240943    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:34.150599 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:34.150569    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:35.945432 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:35.945392    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:36.046516 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:36.046486    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:38.043845 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:38.043806    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7b28t"]
Jul 20 18:22:38.046343 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:38.046321    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7b28t"
Jul 20 18:22:38.048942 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:38.048920    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 20 18:22:38.050142 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:38.050120    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 20 18:22:38.050248 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:38.050120    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-xvpnc\""
Jul 20 18:22:38.055895 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:38.055874    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7b28t"]
Jul 20 18:22:38.181672 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:38.181637    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp7l4\" (UniqueName: \"kubernetes.io/projected/fd8a6550-c238-435f-aedb-0b8a5159f9f6-kube-api-access-bp7l4\") pod \"openshift-lws-operator-fd8ccff4c-7b28t\" (UID: \"fd8a6550-c238-435f-aedb-0b8a5159f9f6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7b28t"
Jul 20 18:22:38.181672 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:38.181672    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd8a6550-c238-435f-aedb-0b8a5159f9f6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-7b28t\" (UID: \"fd8a6550-c238-435f-aedb-0b8a5159f9f6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7b28t"
Jul 20 18:22:38.282494 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:38.282461    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp7l4\" (UniqueName: \"kubernetes.io/projected/fd8a6550-c238-435f-aedb-0b8a5159f9f6-kube-api-access-bp7l4\") pod \"openshift-lws-operator-fd8ccff4c-7b28t\" (UID: \"fd8a6550-c238-435f-aedb-0b8a5159f9f6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7b28t"
Jul 20 18:22:38.282494 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:38.282499    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd8a6550-c238-435f-aedb-0b8a5159f9f6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-7b28t\" (UID: \"fd8a6550-c238-435f-aedb-0b8a5159f9f6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7b28t"
Jul 20 18:22:38.282823 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:38.282807    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd8a6550-c238-435f-aedb-0b8a5159f9f6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-7b28t\" (UID: \"fd8a6550-c238-435f-aedb-0b8a5159f9f6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7b28t"
Jul 20 18:22:38.295501 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:38.295431    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp7l4\" (UniqueName: \"kubernetes.io/projected/fd8a6550-c238-435f-aedb-0b8a5159f9f6-kube-api-access-bp7l4\") pod \"openshift-lws-operator-fd8ccff4c-7b28t\" (UID: \"fd8a6550-c238-435f-aedb-0b8a5159f9f6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7b28t"
Jul 20 18:22:38.367092 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:38.367046    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7b28t"
Jul 20 18:22:38.497964 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:38.497834    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7b28t"]
Jul 20 18:22:38.500245 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:22:38.500220    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd8a6550_c238_435f_aedb_0b8a5159f9f6.slice/crio-a396d6293e07fd9d971b6297f5fde6e3cc2d5c10672895211f26e1be4087d2a0 WatchSource:0}: Error finding container a396d6293e07fd9d971b6297f5fde6e3cc2d5c10672895211f26e1be4087d2a0: Status 404 returned error can't find the container with id a396d6293e07fd9d971b6297f5fde6e3cc2d5c10672895211f26e1be4087d2a0
Jul 20 18:22:38.880221 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:38.880187    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7b28t" event={"ID":"fd8a6550-c238-435f-aedb-0b8a5159f9f6","Type":"ContainerStarted","Data":"a396d6293e07fd9d971b6297f5fde6e3cc2d5c10672895211f26e1be4087d2a0"}
Jul 20 18:22:41.893751 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:41.893713    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7b28t" event={"ID":"fd8a6550-c238-435f-aedb-0b8a5159f9f6","Type":"ContainerStarted","Data":"76f218aef9134fe3349586959780105d40fe3b438faf0f8a24e95ce4f2a7ee22"}
Jul 20 18:22:41.909257 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:41.909210    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7b28t" podStartSLOduration=1.477562443 podStartE2EDuration="3.909193949s" totalImagesPullingTime="2.431631506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:22:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:22:41.908266419 +0000 UTC m=+558.742238827" watchObservedRunningTime="2026-07-20 18:22:41.909193949 +0000 UTC m=+558.743166336"
Jul 20 18:22:42.625896 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:42.625863    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:42.728945 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:42.728910    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:44.675848 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:44.675812    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:52.770744 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:52.770705    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:56.994689 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:56.994649    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:57.085330 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.085287    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"]
Jul 20 18:22:57.087772 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.087760    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"
Jul 20 18:22:57.091354 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.091333    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vh9n9\""
Jul 20 18:22:57.091354 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.091350    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 20 18:22:57.091545 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.091371    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 20 18:22:57.091545 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.091340    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 20 18:22:57.103626 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.103602    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"]
Jul 20 18:22:57.105991 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.105971    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:57.145182 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.145152    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8e4a50a-e606-41d7-a4aa-df23be7712de-cert\") pod \"lws-controller-manager-bdf467c7d-vtdk8\" (UID: \"d8e4a50a-e606-41d7-a4aa-df23be7712de\") " pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"
Jul 20 18:22:57.145333 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.145242    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d8e4a50a-e606-41d7-a4aa-df23be7712de-metrics-cert\") pod \"lws-controller-manager-bdf467c7d-vtdk8\" (UID: \"d8e4a50a-e606-41d7-a4aa-df23be7712de\") " pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"
Jul 20 18:22:57.145333 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.145296    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d8e4a50a-e606-41d7-a4aa-df23be7712de-manager-config\") pod \"lws-controller-manager-bdf467c7d-vtdk8\" (UID: \"d8e4a50a-e606-41d7-a4aa-df23be7712de\") " pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"
Jul 20 18:22:57.145333 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.145330    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbp4j\" (UniqueName: \"kubernetes.io/projected/d8e4a50a-e606-41d7-a4aa-df23be7712de-kube-api-access-xbp4j\") pod \"lws-controller-manager-bdf467c7d-vtdk8\" (UID: \"d8e4a50a-e606-41d7-a4aa-df23be7712de\") " pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"
Jul 20 18:22:57.246589 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.246504    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d8e4a50a-e606-41d7-a4aa-df23be7712de-metrics-cert\") pod \"lws-controller-manager-bdf467c7d-vtdk8\" (UID: \"d8e4a50a-e606-41d7-a4aa-df23be7712de\") " pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"
Jul 20 18:22:57.246589 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.246541    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d8e4a50a-e606-41d7-a4aa-df23be7712de-manager-config\") pod \"lws-controller-manager-bdf467c7d-vtdk8\" (UID: \"d8e4a50a-e606-41d7-a4aa-df23be7712de\") " pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"
Jul 20 18:22:57.246589 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.246559    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbp4j\" (UniqueName: \"kubernetes.io/projected/d8e4a50a-e606-41d7-a4aa-df23be7712de-kube-api-access-xbp4j\") pod \"lws-controller-manager-bdf467c7d-vtdk8\" (UID: \"d8e4a50a-e606-41d7-a4aa-df23be7712de\") " pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"
Jul 20 18:22:57.246876 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.246622    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8e4a50a-e606-41d7-a4aa-df23be7712de-cert\") pod \"lws-controller-manager-bdf467c7d-vtdk8\" (UID: \"d8e4a50a-e606-41d7-a4aa-df23be7712de\") " pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"
Jul 20 18:22:57.247307 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.247282    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d8e4a50a-e606-41d7-a4aa-df23be7712de-manager-config\") pod \"lws-controller-manager-bdf467c7d-vtdk8\" (UID: \"d8e4a50a-e606-41d7-a4aa-df23be7712de\") " pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"
Jul 20 18:22:57.249013 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.248994    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8e4a50a-e606-41d7-a4aa-df23be7712de-cert\") pod \"lws-controller-manager-bdf467c7d-vtdk8\" (UID: \"d8e4a50a-e606-41d7-a4aa-df23be7712de\") " pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"
Jul 20 18:22:57.249178 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.249157    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d8e4a50a-e606-41d7-a4aa-df23be7712de-metrics-cert\") pod \"lws-controller-manager-bdf467c7d-vtdk8\" (UID: \"d8e4a50a-e606-41d7-a4aa-df23be7712de\") " pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"
Jul 20 18:22:57.259013 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.258985    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbp4j\" (UniqueName: \"kubernetes.io/projected/d8e4a50a-e606-41d7-a4aa-df23be7712de-kube-api-access-xbp4j\") pod \"lws-controller-manager-bdf467c7d-vtdk8\" (UID: \"d8e4a50a-e606-41d7-a4aa-df23be7712de\") " pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"
Jul 20 18:22:57.357070 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.357027    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:57.396604 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.396573    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"
Jul 20 18:22:57.455944 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.455911    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:57.528738 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.528716    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"]
Jul 20 18:22:57.531280 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:22:57.531249    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8e4a50a_e606_41d7_a4aa_df23be7712de.slice/crio-da1630584dedabb333cf7ae46c9c2ea75a34d8cfcff585bf65113931d9f90a98 WatchSource:0}: Error finding container da1630584dedabb333cf7ae46c9c2ea75a34d8cfcff585bf65113931d9f90a98: Status 404 returned error can't find the container with id da1630584dedabb333cf7ae46c9c2ea75a34d8cfcff585bf65113931d9f90a98
Jul 20 18:22:57.958707 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.958624    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8" event={"ID":"d8e4a50a-e606-41d7-a4aa-df23be7712de","Type":"ContainerStarted","Data":"da1630584dedabb333cf7ae46c9c2ea75a34d8cfcff585bf65113931d9f90a98"}
Jul 20 18:22:57.985785 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:57.985755    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:58.329271 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.329241    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:58.435261 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.435228    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:58.483821 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.483787    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"]
Jul 20 18:22:58.490740 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.490712    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:58.496918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.496892    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 20 18:22:58.497049 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.496898    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 20 18:22:58.497608 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.497589    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gwrk8\""
Jul 20 18:22:58.497734 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.497714    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 20 18:22:58.500013 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.499632    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 20 18:22:58.517971 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.517945    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"]
Jul 20 18:22:58.556691 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.556660    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aceaa6ef-fd5c-4e3a-861c-f84e98b706b7-webhook-cert\") pod \"opendatahub-operator-controller-manager-9fdd66b9b-xk5ht\" (UID: \"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7\") " pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:58.556842 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.556732    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5j28\" (UniqueName: \"kubernetes.io/projected/aceaa6ef-fd5c-4e3a-861c-f84e98b706b7-kube-api-access-n5j28\") pod \"opendatahub-operator-controller-manager-9fdd66b9b-xk5ht\" (UID: \"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7\") " pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:58.556842 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.556794    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aceaa6ef-fd5c-4e3a-861c-f84e98b706b7-tmp\") pod \"opendatahub-operator-controller-manager-9fdd66b9b-xk5ht\" (UID: \"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7\") " pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:58.556842 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.556818    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aceaa6ef-fd5c-4e3a-861c-f84e98b706b7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9fdd66b9b-xk5ht\" (UID: \"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7\") " pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:58.556987 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.556854    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aceaa6ef-fd5c-4e3a-861c-f84e98b706b7-manifests\") pod \"opendatahub-operator-controller-manager-9fdd66b9b-xk5ht\" (UID: \"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7\") " pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:58.647593 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.647495    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:58.657732 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.657699    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5j28\" (UniqueName: \"kubernetes.io/projected/aceaa6ef-fd5c-4e3a-861c-f84e98b706b7-kube-api-access-n5j28\") pod \"opendatahub-operator-controller-manager-9fdd66b9b-xk5ht\" (UID: \"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7\") " pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:58.657936 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.657751    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aceaa6ef-fd5c-4e3a-861c-f84e98b706b7-tmp\") pod \"opendatahub-operator-controller-manager-9fdd66b9b-xk5ht\" (UID: \"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7\") " pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:58.657936 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.657769    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aceaa6ef-fd5c-4e3a-861c-f84e98b706b7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9fdd66b9b-xk5ht\" (UID: \"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7\") " pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:58.657936 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.657800    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aceaa6ef-fd5c-4e3a-861c-f84e98b706b7-manifests\") pod \"opendatahub-operator-controller-manager-9fdd66b9b-xk5ht\" (UID: \"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7\") " pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:58.657936 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.657837    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aceaa6ef-fd5c-4e3a-861c-f84e98b706b7-webhook-cert\") pod \"opendatahub-operator-controller-manager-9fdd66b9b-xk5ht\" (UID: \"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7\") " pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:58.658331 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.658294    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aceaa6ef-fd5c-4e3a-861c-f84e98b706b7-manifests\") pod \"opendatahub-operator-controller-manager-9fdd66b9b-xk5ht\" (UID: \"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7\") " pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:58.660361 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.660313    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aceaa6ef-fd5c-4e3a-861c-f84e98b706b7-tmp\") pod \"opendatahub-operator-controller-manager-9fdd66b9b-xk5ht\" (UID: \"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7\") " pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:58.660685 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.660665    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aceaa6ef-fd5c-4e3a-861c-f84e98b706b7-webhook-cert\") pod \"opendatahub-operator-controller-manager-9fdd66b9b-xk5ht\" (UID: \"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7\") " pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:58.660765 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.660746    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aceaa6ef-fd5c-4e3a-861c-f84e98b706b7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9fdd66b9b-xk5ht\" (UID: \"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7\") " pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:58.667965 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.667939    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5j28\" (UniqueName: \"kubernetes.io/projected/aceaa6ef-fd5c-4e3a-861c-f84e98b706b7-kube-api-access-n5j28\") pod \"opendatahub-operator-controller-manager-9fdd66b9b-xk5ht\" (UID: \"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7\") " pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:58.776944 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.776911    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:58.803870 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:58.803841    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:22:59.022873 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:59.022839    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"]
Jul 20 18:22:59.026623 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:22:59.026598    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaceaa6ef_fd5c_4e3a_861c_f84e98b706b7.slice/crio-ba27c02dc472a388af9db518a9d2079dd146fda46b77f6b6bd85c824c50945d8 WatchSource:0}: Error finding container ba27c02dc472a388af9db518a9d2079dd146fda46b77f6b6bd85c824c50945d8: Status 404 returned error can't find the container with id ba27c02dc472a388af9db518a9d2079dd146fda46b77f6b6bd85c824c50945d8
Jul 20 18:22:59.073474 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:59.073438    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:59.188744 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:59.188652    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:22:59.969990 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:59.969954    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht" event={"ID":"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7","Type":"ContainerStarted","Data":"ba27c02dc472a388af9db518a9d2079dd146fda46b77f6b6bd85c824c50945d8"}
Jul 20 18:22:59.971625 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:59.971598    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8" event={"ID":"d8e4a50a-e606-41d7-a4aa-df23be7712de","Type":"ContainerStarted","Data":"8b001acd8f108d1ac0c8a979a4882c357db0e459b3dec91f77902eda3030800d"}
Jul 20 18:22:59.971765 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:59.971741    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"
Jul 20 18:22:59.990604 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:22:59.990556    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8" podStartSLOduration=1.5967699290000001 podStartE2EDuration="2.990533694s" totalImagesPullingTime="1.393763765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:22:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:22:59.989114807 +0000 UTC m=+576.823087196" watchObservedRunningTime="2026-07-20 18:22:59.990533694 +0000 UTC m=+576.824506084"
Jul 20 18:23:02.989619 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:02.989587    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht" event={"ID":"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7","Type":"ContainerStarted","Data":"f5505590e0c7533ffcd2c6a96fd53b9859909c2b8a42e4cd47f0890d38bac8ec"}
Jul 20 18:23:03.995337 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:03.995306    2569 generic.go:356] "Generic (PLEG): container finished" podID="aceaa6ef-fd5c-4e3a-861c-f84e98b706b7" containerID="f5505590e0c7533ffcd2c6a96fd53b9859909c2b8a42e4cd47f0890d38bac8ec" exitCode=0
Jul 20 18:23:03.995794 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:03.995390    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht" event={"ID":"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7","Type":"ContainerDied","Data":"f5505590e0c7533ffcd2c6a96fd53b9859909c2b8a42e4cd47f0890d38bac8ec"}
Jul 20 18:23:05.002861 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:05.002822    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht" event={"ID":"aceaa6ef-fd5c-4e3a-861c-f84e98b706b7","Type":"ContainerStarted","Data":"8c36987a4f0061e0ee292b3e1357cdece23260951e280e824c12dc3de2d7804e"}
Jul 20 18:23:05.003288 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:05.002951    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:23:05.025243 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:05.025193    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht" podStartSLOduration=2.381655707 podStartE2EDuration="7.02517593s" totalImagesPullingTime="3.601372149s" totalInitContainerRuntime="1.042148074s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:22:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:23:05.024521113 +0000 UTC m=+581.858493519" watchObservedRunningTime="2026-07-20 18:23:05.02517593 +0000 UTC m=+581.859148318"
Jul 20 18:23:09.577674 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:09.577636    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:09.679475 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:09.679436    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:10.979699 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:10.979668    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-bdf467c7d-vtdk8"
Jul 20 18:23:16.010573 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:16.010545    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9fdd66b9b-xk5ht"
Jul 20 18:23:18.052109 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:18.052073    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:18.172325 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:18.172295    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:19.422982 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:19.422925    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:19.548411 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:19.548378    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:19.676538 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:19.676469    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:19.780443 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:19.780414    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:19.926895 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:19.926813    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7596f649d-zb7zz"]
Jul 20 18:23:19.929414 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:19.929393    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7596f649d-zb7zz"
Jul 20 18:23:19.931809 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:19.931787    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 20 18:23:19.932790 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:19.932766    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 20 18:23:19.932886 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:19.932835    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-bjg7w\""
Jul 20 18:23:19.932949 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:19.932775    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 20 18:23:19.933049 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:19.933021    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 20 18:23:19.945044 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:19.945024    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7596f649d-zb7zz"]
Jul 20 18:23:19.957447 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:19.957420    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:20.049125 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:20.049092    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kx5p\" (UniqueName: \"kubernetes.io/projected/3b5d6211-f6f0-4096-8caf-65b47857da0a-kube-api-access-7kx5p\") pod \"kube-auth-proxy-7596f649d-zb7zz\" (UID: \"3b5d6211-f6f0-4096-8caf-65b47857da0a\") " pod="openshift-ingress/kube-auth-proxy-7596f649d-zb7zz"
Jul 20 18:23:20.049269 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:20.049135    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5d6211-f6f0-4096-8caf-65b47857da0a-tls-certs\") pod \"kube-auth-proxy-7596f649d-zb7zz\" (UID: \"3b5d6211-f6f0-4096-8caf-65b47857da0a\") " pod="openshift-ingress/kube-auth-proxy-7596f649d-zb7zz"
Jul 20 18:23:20.049269 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:20.049165    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b5d6211-f6f0-4096-8caf-65b47857da0a-tmp\") pod \"kube-auth-proxy-7596f649d-zb7zz\" (UID: \"3b5d6211-f6f0-4096-8caf-65b47857da0a\") " pod="openshift-ingress/kube-auth-proxy-7596f649d-zb7zz"
Jul 20 18:23:20.150370 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:20.150340    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b5d6211-f6f0-4096-8caf-65b47857da0a-tmp\") pod \"kube-auth-proxy-7596f649d-zb7zz\" (UID: \"3b5d6211-f6f0-4096-8caf-65b47857da0a\") " pod="openshift-ingress/kube-auth-proxy-7596f649d-zb7zz"
Jul 20 18:23:20.150532 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:20.150425    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kx5p\" (UniqueName: \"kubernetes.io/projected/3b5d6211-f6f0-4096-8caf-65b47857da0a-kube-api-access-7kx5p\") pod \"kube-auth-proxy-7596f649d-zb7zz\" (UID: \"3b5d6211-f6f0-4096-8caf-65b47857da0a\") " pod="openshift-ingress/kube-auth-proxy-7596f649d-zb7zz"
Jul 20 18:23:20.150532 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:20.150451    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5d6211-f6f0-4096-8caf-65b47857da0a-tls-certs\") pod \"kube-auth-proxy-7596f649d-zb7zz\" (UID: \"3b5d6211-f6f0-4096-8caf-65b47857da0a\") " pod="openshift-ingress/kube-auth-proxy-7596f649d-zb7zz"
Jul 20 18:23:20.152807 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:20.152773    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b5d6211-f6f0-4096-8caf-65b47857da0a-tmp\") pod \"kube-auth-proxy-7596f649d-zb7zz\" (UID: \"3b5d6211-f6f0-4096-8caf-65b47857da0a\") " pod="openshift-ingress/kube-auth-proxy-7596f649d-zb7zz"
Jul 20 18:23:20.153012 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:20.152989    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5d6211-f6f0-4096-8caf-65b47857da0a-tls-certs\") pod \"kube-auth-proxy-7596f649d-zb7zz\" (UID: \"3b5d6211-f6f0-4096-8caf-65b47857da0a\") " pod="openshift-ingress/kube-auth-proxy-7596f649d-zb7zz"
Jul 20 18:23:20.162722 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:20.162695    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kx5p\" (UniqueName: \"kubernetes.io/projected/3b5d6211-f6f0-4096-8caf-65b47857da0a-kube-api-access-7kx5p\") pod \"kube-auth-proxy-7596f649d-zb7zz\" (UID: \"3b5d6211-f6f0-4096-8caf-65b47857da0a\") " pod="openshift-ingress/kube-auth-proxy-7596f649d-zb7zz"
Jul 20 18:23:20.207723 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:20.207657    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:20.239451 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:20.239426    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7596f649d-zb7zz"
Jul 20 18:23:20.367945 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:20.367922    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7596f649d-zb7zz"]
Jul 20 18:23:20.370831 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:23:20.370805    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b5d6211_f6f0_4096_8caf_65b47857da0a.slice/crio-a5f1949afda5e3a98886b17a2b7dbc1382ed2bf503bd0174f3d0eb6005341595 WatchSource:0}: Error finding container a5f1949afda5e3a98886b17a2b7dbc1382ed2bf503bd0174f3d0eb6005341595: Status 404 returned error can't find the container with id a5f1949afda5e3a98886b17a2b7dbc1382ed2bf503bd0174f3d0eb6005341595
Jul 20 18:23:20.757165 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:20.757131    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:20.965744 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:20.965711    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:21.038566 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:21.038489    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:21.078364 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:21.078327    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7596f649d-zb7zz" event={"ID":"3b5d6211-f6f0-4096-8caf-65b47857da0a","Type":"ContainerStarted","Data":"a5f1949afda5e3a98886b17a2b7dbc1382ed2bf503bd0174f3d0eb6005341595"}
Jul 20 18:23:21.933790 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:21.933760    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:24.026598 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:24.026579    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 20 18:23:25.100958 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:25.100925    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7596f649d-zb7zz" event={"ID":"3b5d6211-f6f0-4096-8caf-65b47857da0a","Type":"ContainerStarted","Data":"4cff28012eac1b7231e57c32eae6b38df777140f518d0fc8ff0830772573ee75"}
Jul 20 18:23:25.118188 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:25.118143    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7596f649d-zb7zz" podStartSLOduration=2.466726973 podStartE2EDuration="6.118128151s" totalImagesPullingTime="3.651401178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:23:25.116398657 +0000 UTC m=+601.950371046" watchObservedRunningTime="2026-07-20 18:23:25.118128151 +0000 UTC m=+601.952100537"
Jul 20 18:23:26.897739 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:26.897701    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:26.988114 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:26.988082    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:27.145639 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:27.145607    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:27.540428 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:27.540397    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:27.668852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:27.668820    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:30.113400 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:30.113374    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:23:30.113842 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:30.113373    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:23:30.694163 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:30.694116    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:30.800671 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:30.800639    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:30.895607 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:30.895579    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:31.101193 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:31.101166    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:31.963416 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:31.963386    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:32.064855 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:32.064829    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:23:32.143365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:23:32.143329    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:24:10.952371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:24:10.952313    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:24:11.048871 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:24:11.048834    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:24:12.892780 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:24:12.892718    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:24:12.948091 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:24:12.948050    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:24:19.522106 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:24:19.522046    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:24:19.622595 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:24:19.622566    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:24:19.873417 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:24:19.873337    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:24:20.663118 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:24:20.663090    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:24:20.722678 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:24:20.722651    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:24:48.990674 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:24:48.990643    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:24:51.579369 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:24:51.579339    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:24:52.974313 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:24:52.974280    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:24:55.785654 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:24:55.785605    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:24:56.600778 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:24:56.600727    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:24:57.610546 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:24:57.610506    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:24:57.716875 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:24:57.716839    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:00.175498 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:00.175459    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:00.663507 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:00.663469    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:03.884682 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:03.884651    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:05.870184 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:05.870154    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:05.963544 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:05.963511    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:06.265554 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:06.265510    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:06.370666 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:06.370633    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:09.219163 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:09.219129    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:09.322938 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:09.322903    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:11.808373 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:11.808343    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:11.911603 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:11.911570    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:13.520920 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:13.520885    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:13.657318 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:13.657281    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:14.070943 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:14.070907    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:14.889379 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:14.889349    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:14.938817 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:14.938777    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c"]
Jul 20 18:25:14.941258 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:14.941235    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c"
Jul 20 18:25:14.945417 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:14.945393    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 20 18:25:14.945565 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:14.945542    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 20 18:25:14.946376 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:14.946360    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xx5n8\""
Jul 20 18:25:14.946448 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:14.946373    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 20 18:25:14.946693 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:14.946670    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 20 18:25:14.965360 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:14.965337    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c"]
Jul 20 18:25:15.054980 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:15.054900    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/86a0d7ee-3eaf-4cbd-8476-d485b0e85eea-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kd82c\" (UID: \"86a0d7ee-3eaf-4cbd-8476-d485b0e85eea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c"
Jul 20 18:25:15.054980 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:15.054938    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/86a0d7ee-3eaf-4cbd-8476-d485b0e85eea-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kd82c\" (UID: \"86a0d7ee-3eaf-4cbd-8476-d485b0e85eea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c"
Jul 20 18:25:15.055214 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:15.054984    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxz4c\" (UniqueName: \"kubernetes.io/projected/86a0d7ee-3eaf-4cbd-8476-d485b0e85eea-kube-api-access-wxz4c\") pod \"kuadrant-console-plugin-66b5774985-kd82c\" (UID: \"86a0d7ee-3eaf-4cbd-8476-d485b0e85eea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c"
Jul 20 18:25:15.086198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:15.086170    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:15.155935 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:15.155853    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxz4c\" (UniqueName: \"kubernetes.io/projected/86a0d7ee-3eaf-4cbd-8476-d485b0e85eea-kube-api-access-wxz4c\") pod \"kuadrant-console-plugin-66b5774985-kd82c\" (UID: \"86a0d7ee-3eaf-4cbd-8476-d485b0e85eea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c"
Jul 20 18:25:15.155935 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:15.155930    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/86a0d7ee-3eaf-4cbd-8476-d485b0e85eea-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kd82c\" (UID: \"86a0d7ee-3eaf-4cbd-8476-d485b0e85eea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c"
Jul 20 18:25:15.156158 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:15.155955    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/86a0d7ee-3eaf-4cbd-8476-d485b0e85eea-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kd82c\" (UID: \"86a0d7ee-3eaf-4cbd-8476-d485b0e85eea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c"
Jul 20 18:25:15.156561 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:15.156543    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/86a0d7ee-3eaf-4cbd-8476-d485b0e85eea-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kd82c\" (UID: \"86a0d7ee-3eaf-4cbd-8476-d485b0e85eea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c"
Jul 20 18:25:15.158308 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:15.158293    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/86a0d7ee-3eaf-4cbd-8476-d485b0e85eea-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kd82c\" (UID: \"86a0d7ee-3eaf-4cbd-8476-d485b0e85eea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c"
Jul 20 18:25:15.171196 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:15.171170    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxz4c\" (UniqueName: \"kubernetes.io/projected/86a0d7ee-3eaf-4cbd-8476-d485b0e85eea-kube-api-access-wxz4c\") pod \"kuadrant-console-plugin-66b5774985-kd82c\" (UID: \"86a0d7ee-3eaf-4cbd-8476-d485b0e85eea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c"
Jul 20 18:25:15.250175 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:15.250144    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c"
Jul 20 18:25:15.397638 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:15.397613    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c"]
Jul 20 18:25:15.399424 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:25:15.399394    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86a0d7ee_3eaf_4cbd_8476_d485b0e85eea.slice/crio-e4d060d976b335efd49cb87711ef7bdd6c524ade20bb48c2851076338b68c275 WatchSource:0}: Error finding container e4d060d976b335efd49cb87711ef7bdd6c524ade20bb48c2851076338b68c275: Status 404 returned error can't find the container with id e4d060d976b335efd49cb87711ef7bdd6c524ade20bb48c2851076338b68c275
Jul 20 18:25:15.400718 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:15.400701    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 18:25:15.598316 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:15.598280    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c" event={"ID":"86a0d7ee-3eaf-4cbd-8476-d485b0e85eea","Type":"ContainerStarted","Data":"e4d060d976b335efd49cb87711ef7bdd6c524ade20bb48c2851076338b68c275"}
Jul 20 18:25:16.675197 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:16.675144    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:16.791690 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:16.791645    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:17.130729 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:17.130662    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:17.230494 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:17.230461    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:17.550625 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:17.550579    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:17.653594 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:17.653563    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:17.753553 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:17.753519    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:23.891963 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:23.891924    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:24.128968 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:24.128937    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:25.154136 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.154099    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"]
Jul 20 18:25:25.157186 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.157166    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"
Jul 20 18:25:25.160590 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.160564    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-z2ttw\""
Jul 20 18:25:25.171433 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.171407    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"]
Jul 20 18:25:25.247944 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.247908    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdtxs\" (UniqueName: \"kubernetes.io/projected/1eddb6e0-7291-42c2-9cad-1c2035adb25c-kube-api-access-jdtxs\") pod \"kuadrant-operator-controller-manager-55467d9f7-wfz8m\" (UID: \"1eddb6e0-7291-42c2-9cad-1c2035adb25c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"
Jul 20 18:25:25.248171 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.247961    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1eddb6e0-7291-42c2-9cad-1c2035adb25c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-wfz8m\" (UID: \"1eddb6e0-7291-42c2-9cad-1c2035adb25c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"
Jul 20 18:25:25.348900 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.348850    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdtxs\" (UniqueName: \"kubernetes.io/projected/1eddb6e0-7291-42c2-9cad-1c2035adb25c-kube-api-access-jdtxs\") pod \"kuadrant-operator-controller-manager-55467d9f7-wfz8m\" (UID: \"1eddb6e0-7291-42c2-9cad-1c2035adb25c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"
Jul 20 18:25:25.349094 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.348910    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1eddb6e0-7291-42c2-9cad-1c2035adb25c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-wfz8m\" (UID: \"1eddb6e0-7291-42c2-9cad-1c2035adb25c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"
Jul 20 18:25:25.349343 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.349313    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1eddb6e0-7291-42c2-9cad-1c2035adb25c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-wfz8m\" (UID: \"1eddb6e0-7291-42c2-9cad-1c2035adb25c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"
Jul 20 18:25:25.361908 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.361880    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"]
Jul 20 18:25:25.362211 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:25:25.362189    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-jdtxs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m" podUID="1eddb6e0-7291-42c2-9cad-1c2035adb25c"
Jul 20 18:25:25.370913 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.370889    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdtxs\" (UniqueName: \"kubernetes.io/projected/1eddb6e0-7291-42c2-9cad-1c2035adb25c-kube-api-access-jdtxs\") pod \"kuadrant-operator-controller-manager-55467d9f7-wfz8m\" (UID: \"1eddb6e0-7291-42c2-9cad-1c2035adb25c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"
Jul 20 18:25:25.386049 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.386023    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"]
Jul 20 18:25:25.429012 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.428934    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wlm2r"]
Jul 20 18:25:25.431404 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.431386    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wlm2r"
Jul 20 18:25:25.449428 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.449389    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngbj7\" (UniqueName: \"kubernetes.io/projected/d40d847e-2677-4897-b221-60a683a55549-kube-api-access-ngbj7\") pod \"kuadrant-operator-controller-manager-55467d9f7-wlm2r\" (UID: \"d40d847e-2677-4897-b221-60a683a55549\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wlm2r"
Jul 20 18:25:25.449577 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.449433    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d40d847e-2677-4897-b221-60a683a55549-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-wlm2r\" (UID: \"d40d847e-2677-4897-b221-60a683a55549\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wlm2r"
Jul 20 18:25:25.464674 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.464387    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wlm2r"]
Jul 20 18:25:25.495171 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.495130    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wlm2r"]
Jul 20 18:25:25.495545 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:25:25.495509    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-ngbj7], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wlm2r" podUID="d40d847e-2677-4897-b221-60a683a55549"
Jul 20 18:25:25.550030 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.549995    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngbj7\" (UniqueName: \"kubernetes.io/projected/d40d847e-2677-4897-b221-60a683a55549-kube-api-access-ngbj7\") pod \"kuadrant-operator-controller-manager-55467d9f7-wlm2r\" (UID: \"d40d847e-2677-4897-b221-60a683a55549\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wlm2r"
Jul 20 18:25:25.550229 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.550095    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d40d847e-2677-4897-b221-60a683a55549-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-wlm2r\" (UID: \"d40d847e-2677-4897-b221-60a683a55549\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wlm2r"
Jul 20 18:25:25.550560 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.550529    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d40d847e-2677-4897-b221-60a683a55549-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-wlm2r\" (UID: \"d40d847e-2677-4897-b221-60a683a55549\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wlm2r"
Jul 20 18:25:25.575567 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.575532    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngbj7\" (UniqueName: \"kubernetes.io/projected/d40d847e-2677-4897-b221-60a683a55549-kube-api-access-ngbj7\") pod \"kuadrant-operator-controller-manager-55467d9f7-wlm2r\" (UID: \"d40d847e-2677-4897-b221-60a683a55549\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wlm2r"
Jul 20 18:25:25.654714 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.654681    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wlm2r"
Jul 20 18:25:25.654914 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.654681    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"
Jul 20 18:25:25.657225 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:25:25.657195    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-wfz8m\" is forbidden: User \"system:node:ip-10-0-129-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-84.ec2.internal' and this object" podUID="1eddb6e0-7291-42c2-9cad-1c2035adb25c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"
Jul 20 18:25:25.661027 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.661010    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wlm2r"
Jul 20 18:25:25.664559 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.664540    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"
Jul 20 18:25:25.719959 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:25:25.719880    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-wfz8m\" is forbidden: User \"system:node:ip-10-0-129-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-84.ec2.internal' and this object" podUID="1eddb6e0-7291-42c2-9cad-1c2035adb25c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"
Jul 20 18:25:25.752229 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.752192    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1eddb6e0-7291-42c2-9cad-1c2035adb25c-extensions-socket-volume\") pod \"1eddb6e0-7291-42c2-9cad-1c2035adb25c\" (UID: \"1eddb6e0-7291-42c2-9cad-1c2035adb25c\") "
Jul 20 18:25:25.752415 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.752251    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdtxs\" (UniqueName: \"kubernetes.io/projected/1eddb6e0-7291-42c2-9cad-1c2035adb25c-kube-api-access-jdtxs\") pod \"1eddb6e0-7291-42c2-9cad-1c2035adb25c\" (UID: \"1eddb6e0-7291-42c2-9cad-1c2035adb25c\") "
Jul 20 18:25:25.752415 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.752303    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngbj7\" (UniqueName: \"kubernetes.io/projected/d40d847e-2677-4897-b221-60a683a55549-kube-api-access-ngbj7\") pod \"d40d847e-2677-4897-b221-60a683a55549\" (UID: \"d40d847e-2677-4897-b221-60a683a55549\") "
Jul 20 18:25:25.752415 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.752333    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d40d847e-2677-4897-b221-60a683a55549-extensions-socket-volume\") pod \"d40d847e-2677-4897-b221-60a683a55549\" (UID: \"d40d847e-2677-4897-b221-60a683a55549\") "
Jul 20 18:25:25.752593 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.752500    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1eddb6e0-7291-42c2-9cad-1c2035adb25c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1eddb6e0-7291-42c2-9cad-1c2035adb25c" (UID: "1eddb6e0-7291-42c2-9cad-1c2035adb25c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 18:25:25.752680 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.752647    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1eddb6e0-7291-42c2-9cad-1c2035adb25c-extensions-socket-volume\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:25:25.752680 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.752654    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d40d847e-2677-4897-b221-60a683a55549-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d40d847e-2677-4897-b221-60a683a55549" (UID: "d40d847e-2677-4897-b221-60a683a55549"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 18:25:25.754761 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.754737    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d40d847e-2677-4897-b221-60a683a55549-kube-api-access-ngbj7" (OuterVolumeSpecName: "kube-api-access-ngbj7") pod "d40d847e-2677-4897-b221-60a683a55549" (UID: "d40d847e-2677-4897-b221-60a683a55549"). InnerVolumeSpecName "kube-api-access-ngbj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:25:25.755316 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.755289    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1eddb6e0-7291-42c2-9cad-1c2035adb25c-kube-api-access-jdtxs" (OuterVolumeSpecName: "kube-api-access-jdtxs") pod "1eddb6e0-7291-42c2-9cad-1c2035adb25c" (UID: "1eddb6e0-7291-42c2-9cad-1c2035adb25c"). InnerVolumeSpecName "kube-api-access-jdtxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:25:25.760185 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.760114    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1eddb6e0-7291-42c2-9cad-1c2035adb25c" path="/var/lib/kubelet/pods/1eddb6e0-7291-42c2-9cad-1c2035adb25c/volumes"
Jul 20 18:25:25.853471 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.853432    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdtxs\" (UniqueName: \"kubernetes.io/projected/1eddb6e0-7291-42c2-9cad-1c2035adb25c-kube-api-access-jdtxs\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:25:25.853471 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.853460    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngbj7\" (UniqueName: \"kubernetes.io/projected/d40d847e-2677-4897-b221-60a683a55549-kube-api-access-ngbj7\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:25:25.853471 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:25.853476    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d40d847e-2677-4897-b221-60a683a55549-extensions-socket-volume\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:25:26.660349 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:26.660316    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"
Jul 20 18:25:26.660349 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:26.660327    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wlm2r"
Jul 20 18:25:26.667658 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:25:26.667607    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-wfz8m\" is forbidden: User \"system:node:ip-10-0-129-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-84.ec2.internal' and this object" podUID="1eddb6e0-7291-42c2-9cad-1c2035adb25c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wfz8m"
Jul 20 18:25:26.718952 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:26.718923    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wlm2r"]
Jul 20 18:25:26.725350 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:26.725312    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wlm2r"]
Jul 20 18:25:26.993714 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:26.993675    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:27.106454 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:27.106424    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:27.759457 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:27.759423    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d40d847e-2677-4897-b221-60a683a55549" path="/var/lib/kubelet/pods/d40d847e-2677-4897-b221-60a683a55549/volumes"
Jul 20 18:25:31.557303 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:31.557259    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:31.728071 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:31.727999    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:31.825483 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:31.825395    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:36.938932 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:36.938851    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:37.079298 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:37.079248    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:41.740482 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:41.740442    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c" event={"ID":"86a0d7ee-3eaf-4cbd-8476-d485b0e85eea","Type":"ContainerStarted","Data":"3e5ffd44973d2a51c8e2cb4e901bc5375b672cd5ae2d7b17d54bab791f4ae1cb"}
Jul 20 18:25:41.766790 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:41.766740    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd82c" podStartSLOduration=2.510512184 podStartE2EDuration="27.766726165s" totalImagesPullingTime="25.256213981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:25:41.765762825 +0000 UTC m=+738.599735214" watchObservedRunningTime="2026-07-20 18:25:41.766726165 +0000 UTC m=+738.600698553"
Jul 20 18:25:54.081601 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:54.081572    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:54.341979 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:54.341862    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:56.126240 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.126208    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:56.135641 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.135616    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-drjmq"]
Jul 20 18:25:56.138330 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.138313    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq"
Jul 20 18:25:56.141357 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.141336    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 20 18:25:56.157335 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.157295    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-drjmq"]
Jul 20 18:25:56.224277 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.224244    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-drjmq"]
Jul 20 18:25:56.226749 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.226732    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6-config-file\") pod \"limitador-limitador-64c94fd64f-drjmq\" (UID: \"4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq"
Jul 20 18:25:56.226817 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.226770    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w4vv\" (UniqueName: \"kubernetes.io/projected/4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6-kube-api-access-7w4vv\") pod \"limitador-limitador-64c94fd64f-drjmq\" (UID: \"4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq"
Jul 20 18:25:56.228748 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.228722    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:56.327583 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.327554    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6-config-file\") pod \"limitador-limitador-64c94fd64f-drjmq\" (UID: \"4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq"
Jul 20 18:25:56.327761 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.327604    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w4vv\" (UniqueName: \"kubernetes.io/projected/4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6-kube-api-access-7w4vv\") pod \"limitador-limitador-64c94fd64f-drjmq\" (UID: \"4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq"
Jul 20 18:25:56.328217 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.328199    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6-config-file\") pod \"limitador-limitador-64c94fd64f-drjmq\" (UID: \"4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq"
Jul 20 18:25:56.345962 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.345929    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w4vv\" (UniqueName: \"kubernetes.io/projected/4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6-kube-api-access-7w4vv\") pod \"limitador-limitador-64c94fd64f-drjmq\" (UID: \"4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq"
Jul 20 18:25:56.350649 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.350624    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:56.448083 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.447969    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq"
Jul 20 18:25:56.594552 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.594518    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-drjmq"]
Jul 20 18:25:56.597404 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:25:56.597382    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a3b9be1_ecc4_4a58_9e7c_35f8f3a81fd6.slice/crio-d3cd8451a6650df0364f190ed8984dede72508c5af414d500328b510e438cc6f WatchSource:0}: Error finding container d3cd8451a6650df0364f190ed8984dede72508c5af414d500328b510e438cc6f: Status 404 returned error can't find the container with id d3cd8451a6650df0364f190ed8984dede72508c5af414d500328b510e438cc6f
Jul 20 18:25:56.813110 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:56.813076    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq" event={"ID":"4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6","Type":"ContainerStarted","Data":"d3cd8451a6650df0364f190ed8984dede72508c5af414d500328b510e438cc6f"}
Jul 20 18:25:57.121793 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:57.121718    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:57.217984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:57.217947    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:25:57.278534 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:57.278503    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-nlrsf"]
Jul 20 18:25:57.281138 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:57.281116    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-nlrsf"
Jul 20 18:25:57.284497 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:57.284476    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-92m5b\""
Jul 20 18:25:57.292717 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:57.292693    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nlrsf"]
Jul 20 18:25:57.437160 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:57.436947    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfw6z\" (UniqueName: \"kubernetes.io/projected/25534499-4b30-4550-9334-1bb7e5514ee0-kube-api-access-hfw6z\") pod \"authorino-75d556bd77-nlrsf\" (UID: \"25534499-4b30-4550-9334-1bb7e5514ee0\") " pod="kuadrant-system/authorino-75d556bd77-nlrsf"
Jul 20 18:25:57.537790 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:57.537746    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfw6z\" (UniqueName: \"kubernetes.io/projected/25534499-4b30-4550-9334-1bb7e5514ee0-kube-api-access-hfw6z\") pod \"authorino-75d556bd77-nlrsf\" (UID: \"25534499-4b30-4550-9334-1bb7e5514ee0\") " pod="kuadrant-system/authorino-75d556bd77-nlrsf"
Jul 20 18:25:57.552350 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:57.552318    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfw6z\" (UniqueName: \"kubernetes.io/projected/25534499-4b30-4550-9334-1bb7e5514ee0-kube-api-access-hfw6z\") pod \"authorino-75d556bd77-nlrsf\" (UID: \"25534499-4b30-4550-9334-1bb7e5514ee0\") " pod="kuadrant-system/authorino-75d556bd77-nlrsf"
Jul 20 18:25:57.594274 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:57.594171    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-nlrsf"
Jul 20 18:25:57.767260 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:57.767227    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nlrsf"]
Jul 20 18:25:57.771363 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:25:57.771329    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25534499_4b30_4550_9334_1bb7e5514ee0.slice/crio-883626948d9c4276d1d4feb38adee0df272e4b6f9e14207105402e9932ea1239 WatchSource:0}: Error finding container 883626948d9c4276d1d4feb38adee0df272e4b6f9e14207105402e9932ea1239: Status 404 returned error can't find the container with id 883626948d9c4276d1d4feb38adee0df272e4b6f9e14207105402e9932ea1239
Jul 20 18:25:57.820683 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:57.820639    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-nlrsf" event={"ID":"25534499-4b30-4550-9334-1bb7e5514ee0","Type":"ContainerStarted","Data":"883626948d9c4276d1d4feb38adee0df272e4b6f9e14207105402e9932ea1239"}
Jul 20 18:25:59.834934 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:59.834864    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq" event={"ID":"4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6","Type":"ContainerStarted","Data":"c77a48ae2b09be42469816ca0f87559fc8126be212cafb6945681d21129e4803"}
Jul 20 18:25:59.835434 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:59.835023    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq"
Jul 20 18:25:59.856893 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:25:59.856796    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq" podStartSLOduration=1.2058114739999999 podStartE2EDuration="3.85677688s" totalImagesPullingTime="2.650965406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:25:59.855534261 +0000 UTC m=+756.689506682" watchObservedRunningTime="2026-07-20 18:25:59.85677688 +0000 UTC m=+756.690749268"
Jul 20 18:26:01.846552 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:01.846448    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-nlrsf" event={"ID":"25534499-4b30-4550-9334-1bb7e5514ee0","Type":"ContainerStarted","Data":"e67f2902e5b153bf2d833e35f5115fd4d367f1b37f67a65fcaafadf0af69e8c9"}
Jul 20 18:26:01.867633 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:01.867578    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-nlrsf" podStartSLOduration=1.108799966 podStartE2EDuration="4.867559118s" totalImagesPullingTime="3.758759152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:26:01.8665441 +0000 UTC m=+758.700516489" watchObservedRunningTime="2026-07-20 18:26:01.867559118 +0000 UTC m=+758.701531506"
Jul 20 18:26:10.840744 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:10.840715    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq"
Jul 20 18:26:11.657099 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:11.657041    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-drjmq"]
Jul 20 18:26:11.657281 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:11.657239    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq" podUID="4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6" containerName="limitador" containerID="cri-o://c77a48ae2b09be42469816ca0f87559fc8126be212cafb6945681d21129e4803" gracePeriod=30
Jul 20 18:26:12.261676 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:12.261652    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq"
Jul 20 18:26:12.273403 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:12.273381    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6-config-file\") pod \"4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6\" (UID: \"4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6\") "
Jul 20 18:26:12.273499 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:12.273452    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w4vv\" (UniqueName: \"kubernetes.io/projected/4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6-kube-api-access-7w4vv\") pod \"4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6\" (UID: \"4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6\") "
Jul 20 18:26:12.273787 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:12.273755    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6-config-file" (OuterVolumeSpecName: "config-file") pod "4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6" (UID: "4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:26:12.275480 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:12.275460    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6-kube-api-access-7w4vv" (OuterVolumeSpecName: "kube-api-access-7w4vv") pod "4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6" (UID: "4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6"). InnerVolumeSpecName "kube-api-access-7w4vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:26:12.374647 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:12.374606    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w4vv\" (UniqueName: \"kubernetes.io/projected/4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6-kube-api-access-7w4vv\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:26:12.374647 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:12.374638    2569 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6-config-file\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:26:12.904904 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:12.904866    2569 generic.go:356] "Generic (PLEG): container finished" podID="4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6" containerID="c77a48ae2b09be42469816ca0f87559fc8126be212cafb6945681d21129e4803" exitCode=0
Jul 20 18:26:12.905109 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:12.904913    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq"
Jul 20 18:26:12.905109 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:12.904954    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq" event={"ID":"4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6","Type":"ContainerDied","Data":"c77a48ae2b09be42469816ca0f87559fc8126be212cafb6945681d21129e4803"}
Jul 20 18:26:12.905109 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:12.905011    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-drjmq" event={"ID":"4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6","Type":"ContainerDied","Data":"d3cd8451a6650df0364f190ed8984dede72508c5af414d500328b510e438cc6f"}
Jul 20 18:26:12.905109 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:12.905033    2569 scope.go:117] "RemoveContainer" containerID="c77a48ae2b09be42469816ca0f87559fc8126be212cafb6945681d21129e4803"
Jul 20 18:26:12.913666 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:12.913650    2569 scope.go:117] "RemoveContainer" containerID="c77a48ae2b09be42469816ca0f87559fc8126be212cafb6945681d21129e4803"
Jul 20 18:26:12.913907 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:26:12.913891    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c77a48ae2b09be42469816ca0f87559fc8126be212cafb6945681d21129e4803\": container with ID starting with c77a48ae2b09be42469816ca0f87559fc8126be212cafb6945681d21129e4803 not found: ID does not exist" containerID="c77a48ae2b09be42469816ca0f87559fc8126be212cafb6945681d21129e4803"
Jul 20 18:26:12.913975 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:12.913915    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c77a48ae2b09be42469816ca0f87559fc8126be212cafb6945681d21129e4803"} err="failed to get container status \"c77a48ae2b09be42469816ca0f87559fc8126be212cafb6945681d21129e4803\": rpc error: code = NotFound desc = could not find container \"c77a48ae2b09be42469816ca0f87559fc8126be212cafb6945681d21129e4803\": container with ID starting with c77a48ae2b09be42469816ca0f87559fc8126be212cafb6945681d21129e4803 not found: ID does not exist"
Jul 20 18:26:12.928535 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:12.928509    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-drjmq"]
Jul 20 18:26:12.931253 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:12.931232    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-drjmq"]
Jul 20 18:26:13.758268 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:13.758237    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6" path="/var/lib/kubelet/pods/4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6/volumes"
Jul 20 18:26:16.162453 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:16.162410    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:26:17.405619 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:17.405582    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:26:17.498516 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:17.498484    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:26:17.890672 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:17.890638    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8rm4p"]
Jul 20 18:26:17.891029 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:17.891017    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6" containerName="limitador"
Jul 20 18:26:17.891095 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:17.891033    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6" containerName="limitador"
Jul 20 18:26:17.891146 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:17.891129    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a3b9be1-ecc4-4a58-9e7c-35f8f3a81fd6" containerName="limitador"
Jul 20 18:26:17.895418 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:17.895402    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8rm4p"
Jul 20 18:26:17.901637 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:17.901613    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-8hq6k\""
Jul 20 18:26:17.902105 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:17.902085    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 20 18:26:17.902367 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:17.902349    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 18:26:17.904416 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:17.904397    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 20 18:26:17.916354 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:17.916330    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8rm4p"]
Jul 20 18:26:17.924880 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:17.924857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8aab38f5-c695-4462-9967-235d657071ac-data\") pod \"postgres-69fb4594cf-8rm4p\" (UID: \"8aab38f5-c695-4462-9967-235d657071ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8rm4p"
Jul 20 18:26:17.925001 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:17.924888    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d58tv\" (UniqueName: \"kubernetes.io/projected/8aab38f5-c695-4462-9967-235d657071ac-kube-api-access-d58tv\") pod \"postgres-69fb4594cf-8rm4p\" (UID: \"8aab38f5-c695-4462-9967-235d657071ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8rm4p"
Jul 20 18:26:17.946942 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:17.946911    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:26:18.025783 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:18.025748    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8aab38f5-c695-4462-9967-235d657071ac-data\") pod \"postgres-69fb4594cf-8rm4p\" (UID: \"8aab38f5-c695-4462-9967-235d657071ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8rm4p"
Jul 20 18:26:18.025783 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:18.025784    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d58tv\" (UniqueName: \"kubernetes.io/projected/8aab38f5-c695-4462-9967-235d657071ac-kube-api-access-d58tv\") pod \"postgres-69fb4594cf-8rm4p\" (UID: \"8aab38f5-c695-4462-9967-235d657071ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8rm4p"
Jul 20 18:26:18.026205 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:18.026187    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8aab38f5-c695-4462-9967-235d657071ac-data\") pod \"postgres-69fb4594cf-8rm4p\" (UID: \"8aab38f5-c695-4462-9967-235d657071ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8rm4p"
Jul 20 18:26:18.035359 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:18.035331    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d58tv\" (UniqueName: \"kubernetes.io/projected/8aab38f5-c695-4462-9967-235d657071ac-kube-api-access-d58tv\") pod \"postgres-69fb4594cf-8rm4p\" (UID: \"8aab38f5-c695-4462-9967-235d657071ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8rm4p"
Jul 20 18:26:18.205766 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:18.205650    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8rm4p"
Jul 20 18:26:18.335230 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:18.335204    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8rm4p"]
Jul 20 18:26:18.337972 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:26:18.337945    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aab38f5_c695_4462_9967_235d657071ac.slice/crio-d8f39c952f1d42e9a79543c5c1b4ac50a3e42db3f9ad8ea11afeb6ac88cb55f0 WatchSource:0}: Error finding container d8f39c952f1d42e9a79543c5c1b4ac50a3e42db3f9ad8ea11afeb6ac88cb55f0: Status 404 returned error can't find the container with id d8f39c952f1d42e9a79543c5c1b4ac50a3e42db3f9ad8ea11afeb6ac88cb55f0
Jul 20 18:26:18.937842 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:18.937804    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8rm4p" event={"ID":"8aab38f5-c695-4462-9967-235d657071ac","Type":"ContainerStarted","Data":"d8f39c952f1d42e9a79543c5c1b4ac50a3e42db3f9ad8ea11afeb6ac88cb55f0"}
Jul 20 18:26:22.756842 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:22.756771    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:26:22.834965 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:22.834927    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:26:23.965706 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:23.965612    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8rm4p" event={"ID":"8aab38f5-c695-4462-9967-235d657071ac","Type":"ContainerStarted","Data":"2ea4bbf2b9c9df3041e49623d7dbead3fdf48960dab80a7ae069acad994990f0"}
Jul 20 18:26:23.966078 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:23.965711    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8rm4p"
Jul 20 18:26:30.003405 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:30.003379    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8rm4p"
Jul 20 18:26:30.037906 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:30.037855    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8rm4p" podStartSLOduration=8.398301751 podStartE2EDuration="13.037840024s" totalImagesPullingTime="4.639538273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:26:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:26:23.987132434 +0000 UTC m=+780.821104822" watchObservedRunningTime="2026-07-20 18:26:30.037840024 +0000 UTC m=+786.871812411"
Jul 20 18:26:30.501370 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:30.501319    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:26:30.590998 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:30.590964    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:26:30.656686 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:30.656653    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4hvjk"]
Jul 20 18:26:30.668242 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:30.668215    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4hvjk"
Jul 20 18:26:30.671958 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:30.671931    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4hvjk"]
Jul 20 18:26:30.750854 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:30.750814    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq4v8\" (UniqueName: \"kubernetes.io/projected/10e9d52d-7533-4fd2-80f5-89e063e243bc-kube-api-access-mq4v8\") pod \"authorino-5d4b67c446-4hvjk\" (UID: \"10e9d52d-7533-4fd2-80f5-89e063e243bc\") " pod="kuadrant-system/authorino-5d4b67c446-4hvjk"
Jul 20 18:26:30.851846 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:30.851760    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq4v8\" (UniqueName: \"kubernetes.io/projected/10e9d52d-7533-4fd2-80f5-89e063e243bc-kube-api-access-mq4v8\") pod \"authorino-5d4b67c446-4hvjk\" (UID: \"10e9d52d-7533-4fd2-80f5-89e063e243bc\") " pod="kuadrant-system/authorino-5d4b67c446-4hvjk"
Jul 20 18:26:30.871073 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:30.871036    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq4v8\" (UniqueName: \"kubernetes.io/projected/10e9d52d-7533-4fd2-80f5-89e063e243bc-kube-api-access-mq4v8\") pod \"authorino-5d4b67c446-4hvjk\" (UID: \"10e9d52d-7533-4fd2-80f5-89e063e243bc\") " pod="kuadrant-system/authorino-5d4b67c446-4hvjk"
Jul 20 18:26:30.938937 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:30.938892    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4hvjk"]
Jul 20 18:26:30.939199 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:30.939184    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4hvjk"
Jul 20 18:26:31.109079 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:31.109036    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4hvjk"]
Jul 20 18:26:31.110887 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:26:31.110851    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10e9d52d_7533_4fd2_80f5_89e063e243bc.slice/crio-b35ed03de3119eec963187e4755320d707e4dd4281ab078d4e4d315085cdcb60 WatchSource:0}: Error finding container b35ed03de3119eec963187e4755320d707e4dd4281ab078d4e4d315085cdcb60: Status 404 returned error can't find the container with id b35ed03de3119eec963187e4755320d707e4dd4281ab078d4e4d315085cdcb60
Jul 20 18:26:32.011308 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:32.011273    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4hvjk" event={"ID":"10e9d52d-7533-4fd2-80f5-89e063e243bc","Type":"ContainerStarted","Data":"1b9fe4483896249620193023659580e7bf5aa147bad94b593b5ece0269dd953c"}
Jul 20 18:26:32.011564 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:32.011319    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4hvjk" event={"ID":"10e9d52d-7533-4fd2-80f5-89e063e243bc","Type":"ContainerStarted","Data":"b35ed03de3119eec963187e4755320d707e4dd4281ab078d4e4d315085cdcb60"}
Jul 20 18:26:32.011564 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:32.011280    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-4hvjk" podUID="10e9d52d-7533-4fd2-80f5-89e063e243bc" containerName="authorino" containerID="cri-o://1b9fe4483896249620193023659580e7bf5aa147bad94b593b5ece0269dd953c" gracePeriod=30
Jul 20 18:26:32.032714 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:32.032667    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-4hvjk" podStartSLOduration=1.583130597 podStartE2EDuration="2.032650541s" totalImagesPullingTime="449.519944ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:26:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:26:32.031393066 +0000 UTC m=+788.865365453" watchObservedRunningTime="2026-07-20 18:26:32.032650541 +0000 UTC m=+788.866622962"
Jul 20 18:26:32.290667 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:32.290583    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4hvjk"
Jul 20 18:26:32.364511 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:32.364477    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mq4v8\" (UniqueName: \"kubernetes.io/projected/10e9d52d-7533-4fd2-80f5-89e063e243bc-kube-api-access-mq4v8\") pod \"10e9d52d-7533-4fd2-80f5-89e063e243bc\" (UID: \"10e9d52d-7533-4fd2-80f5-89e063e243bc\") "
Jul 20 18:26:32.366520 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:32.366496    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10e9d52d-7533-4fd2-80f5-89e063e243bc-kube-api-access-mq4v8" (OuterVolumeSpecName: "kube-api-access-mq4v8") pod "10e9d52d-7533-4fd2-80f5-89e063e243bc" (UID: "10e9d52d-7533-4fd2-80f5-89e063e243bc"). InnerVolumeSpecName "kube-api-access-mq4v8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:26:32.465750 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:32.465708    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq4v8\" (UniqueName: \"kubernetes.io/projected/10e9d52d-7533-4fd2-80f5-89e063e243bc-kube-api-access-mq4v8\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:26:32.836335 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:32.836302    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nlrsf"]
Jul 20 18:26:32.836497 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:32.836471    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-nlrsf" podUID="25534499-4b30-4550-9334-1bb7e5514ee0" containerName="authorino" containerID="cri-o://e67f2902e5b153bf2d833e35f5115fd4d367f1b37f67a65fcaafadf0af69e8c9" gracePeriod=30
Jul 20 18:26:33.019530 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:33.019497    2569 generic.go:356] "Generic (PLEG): container finished" podID="25534499-4b30-4550-9334-1bb7e5514ee0" containerID="e67f2902e5b153bf2d833e35f5115fd4d367f1b37f67a65fcaafadf0af69e8c9" exitCode=0
Jul 20 18:26:33.019674 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:33.019573    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-nlrsf" event={"ID":"25534499-4b30-4550-9334-1bb7e5514ee0","Type":"ContainerDied","Data":"e67f2902e5b153bf2d833e35f5115fd4d367f1b37f67a65fcaafadf0af69e8c9"}
Jul 20 18:26:33.020954 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:33.020929    2569 generic.go:356] "Generic (PLEG): container finished" podID="10e9d52d-7533-4fd2-80f5-89e063e243bc" containerID="1b9fe4483896249620193023659580e7bf5aa147bad94b593b5ece0269dd953c" exitCode=0
Jul 20 18:26:33.021085 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:33.020974    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4hvjk"
Jul 20 18:26:33.021085 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:33.021003    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4hvjk" event={"ID":"10e9d52d-7533-4fd2-80f5-89e063e243bc","Type":"ContainerDied","Data":"1b9fe4483896249620193023659580e7bf5aa147bad94b593b5ece0269dd953c"}
Jul 20 18:26:33.021085 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:33.021043    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4hvjk" event={"ID":"10e9d52d-7533-4fd2-80f5-89e063e243bc","Type":"ContainerDied","Data":"b35ed03de3119eec963187e4755320d707e4dd4281ab078d4e4d315085cdcb60"}
Jul 20 18:26:33.021085 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:33.021080    2569 scope.go:117] "RemoveContainer" containerID="1b9fe4483896249620193023659580e7bf5aa147bad94b593b5ece0269dd953c"
Jul 20 18:26:33.030898 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:33.030875    2569 scope.go:117] "RemoveContainer" containerID="1b9fe4483896249620193023659580e7bf5aa147bad94b593b5ece0269dd953c"
Jul 20 18:26:33.031170 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:26:33.031150    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b9fe4483896249620193023659580e7bf5aa147bad94b593b5ece0269dd953c\": container with ID starting with 1b9fe4483896249620193023659580e7bf5aa147bad94b593b5ece0269dd953c not found: ID does not exist" containerID="1b9fe4483896249620193023659580e7bf5aa147bad94b593b5ece0269dd953c"
Jul 20 18:26:33.031278 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:33.031182    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b9fe4483896249620193023659580e7bf5aa147bad94b593b5ece0269dd953c"} err="failed to get container status \"1b9fe4483896249620193023659580e7bf5aa147bad94b593b5ece0269dd953c\": rpc error: code = NotFound desc = could not find container \"1b9fe4483896249620193023659580e7bf5aa147bad94b593b5ece0269dd953c\": container with ID starting with 1b9fe4483896249620193023659580e7bf5aa147bad94b593b5ece0269dd953c not found: ID does not exist"
Jul 20 18:26:33.049101 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:33.049072    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4hvjk"]
Jul 20 18:26:33.056834 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:33.056803    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4hvjk"]
Jul 20 18:26:33.138526 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:33.138451    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-nlrsf"
Jul 20 18:26:33.273630 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:33.273596    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfw6z\" (UniqueName: \"kubernetes.io/projected/25534499-4b30-4550-9334-1bb7e5514ee0-kube-api-access-hfw6z\") pod \"25534499-4b30-4550-9334-1bb7e5514ee0\" (UID: \"25534499-4b30-4550-9334-1bb7e5514ee0\") "
Jul 20 18:26:33.275621 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:33.275599    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25534499-4b30-4550-9334-1bb7e5514ee0-kube-api-access-hfw6z" (OuterVolumeSpecName: "kube-api-access-hfw6z") pod "25534499-4b30-4550-9334-1bb7e5514ee0" (UID: "25534499-4b30-4550-9334-1bb7e5514ee0"). InnerVolumeSpecName "kube-api-access-hfw6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:26:33.374413 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:33.374375    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfw6z\" (UniqueName: \"kubernetes.io/projected/25534499-4b30-4550-9334-1bb7e5514ee0-kube-api-access-hfw6z\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:26:33.758658 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:33.758625    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10e9d52d-7533-4fd2-80f5-89e063e243bc" path="/var/lib/kubelet/pods/10e9d52d-7533-4fd2-80f5-89e063e243bc/volumes"
Jul 20 18:26:34.026550 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:34.026471    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-nlrsf"
Jul 20 18:26:34.026703 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:34.026472    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-nlrsf" event={"ID":"25534499-4b30-4550-9334-1bb7e5514ee0","Type":"ContainerDied","Data":"883626948d9c4276d1d4feb38adee0df272e4b6f9e14207105402e9932ea1239"}
Jul 20 18:26:34.026703 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:34.026586    2569 scope.go:117] "RemoveContainer" containerID="e67f2902e5b153bf2d833e35f5115fd4d367f1b37f67a65fcaafadf0af69e8c9"
Jul 20 18:26:34.061255 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:34.061215    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nlrsf"]
Jul 20 18:26:34.064873 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:34.064844    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nlrsf"]
Jul 20 18:26:35.758442 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:35.758402    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25534499-4b30-4550-9334-1bb7e5514ee0" path="/var/lib/kubelet/pods/25534499-4b30-4550-9334-1bb7e5514ee0/volumes"
Jul 20 18:26:37.914160 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:37.914126    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:26:37.926314 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:37.926280    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-86d7bc6697-d87bw"]
Jul 20 18:26:37.926654 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:37.926642    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10e9d52d-7533-4fd2-80f5-89e063e243bc" containerName="authorino"
Jul 20 18:26:37.926697 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:37.926655    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="10e9d52d-7533-4fd2-80f5-89e063e243bc" containerName="authorino"
Jul 20 18:26:37.926697 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:37.926673    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25534499-4b30-4550-9334-1bb7e5514ee0" containerName="authorino"
Jul 20 18:26:37.926697 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:37.926678    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="25534499-4b30-4550-9334-1bb7e5514ee0" containerName="authorino"
Jul 20 18:26:37.926790 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:37.926747    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="25534499-4b30-4550-9334-1bb7e5514ee0" containerName="authorino"
Jul 20 18:26:37.926790 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:37.926757    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="10e9d52d-7533-4fd2-80f5-89e063e243bc" containerName="authorino"
Jul 20 18:26:37.942079 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:37.942028    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d7bc6697-d87bw"
Jul 20 18:26:37.945466 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:37.945444    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 20 18:26:37.945592 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:37.945488    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 20 18:26:37.945720 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:37.945705    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-94fp5\""
Jul 20 18:26:37.961643 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:37.961613    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86d7bc6697-d87bw"]
Jul 20 18:26:38.014015 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.013985    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c6ws\" (UniqueName: \"kubernetes.io/projected/f504946f-dd3f-4362-b9e0-064fa3fa6036-kube-api-access-9c6ws\") pod \"maas-controller-86d7bc6697-d87bw\" (UID: \"f504946f-dd3f-4362-b9e0-064fa3fa6036\") " pod="opendatahub/maas-controller-86d7bc6697-d87bw"
Jul 20 18:26:38.014194 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.014034    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f504946f-dd3f-4362-b9e0-064fa3fa6036-webhook-cert\") pod \"maas-controller-86d7bc6697-d87bw\" (UID: \"f504946f-dd3f-4362-b9e0-064fa3fa6036\") " pod="opendatahub/maas-controller-86d7bc6697-d87bw"
Jul 20 18:26:38.046724 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.046690    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:26:38.115334 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.115295    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f504946f-dd3f-4362-b9e0-064fa3fa6036-webhook-cert\") pod \"maas-controller-86d7bc6697-d87bw\" (UID: \"f504946f-dd3f-4362-b9e0-064fa3fa6036\") " pod="opendatahub/maas-controller-86d7bc6697-d87bw"
Jul 20 18:26:38.115537 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.115413    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c6ws\" (UniqueName: \"kubernetes.io/projected/f504946f-dd3f-4362-b9e0-064fa3fa6036-kube-api-access-9c6ws\") pod \"maas-controller-86d7bc6697-d87bw\" (UID: \"f504946f-dd3f-4362-b9e0-064fa3fa6036\") " pod="opendatahub/maas-controller-86d7bc6697-d87bw"
Jul 20 18:26:38.115537 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:26:38.115443    2569 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 20 18:26:38.115537 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:26:38.115527    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f504946f-dd3f-4362-b9e0-064fa3fa6036-webhook-cert podName:f504946f-dd3f-4362-b9e0-064fa3fa6036 nodeName:}" failed. No retries permitted until 2026-07-20 18:26:38.615509714 +0000 UTC m=+795.449482102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/f504946f-dd3f-4362-b9e0-064fa3fa6036-webhook-cert") pod "maas-controller-86d7bc6697-d87bw" (UID: "f504946f-dd3f-4362-b9e0-064fa3fa6036") : secret "maas-controller-webhook-cert" not found
Jul 20 18:26:38.132045 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.132020    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c6ws\" (UniqueName: \"kubernetes.io/projected/f504946f-dd3f-4362-b9e0-064fa3fa6036-kube-api-access-9c6ws\") pod \"maas-controller-86d7bc6697-d87bw\" (UID: \"f504946f-dd3f-4362-b9e0-064fa3fa6036\") " pod="opendatahub/maas-controller-86d7bc6697-d87bw"
Jul 20 18:26:38.165933 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.165853    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bf49cccc9-njt5f"]
Jul 20 18:26:38.168834 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.168815    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bf49cccc9-njt5f"
Jul 20 18:26:38.191339 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.191313    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bf49cccc9-njt5f"]
Jul 20 18:26:38.193275 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.193245    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:26:38.282020 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.281990    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:26:38.317008 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.316976    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/980c1960-d168-43a0-8e4a-f673eda04cf0-webhook-cert\") pod \"maas-controller-7bf49cccc9-njt5f\" (UID: \"980c1960-d168-43a0-8e4a-f673eda04cf0\") " pod="opendatahub/maas-controller-7bf49cccc9-njt5f"
Jul 20 18:26:38.317180 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.317080    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk8vg\" (UniqueName: \"kubernetes.io/projected/980c1960-d168-43a0-8e4a-f673eda04cf0-kube-api-access-lk8vg\") pod \"maas-controller-7bf49cccc9-njt5f\" (UID: \"980c1960-d168-43a0-8e4a-f673eda04cf0\") " pod="opendatahub/maas-controller-7bf49cccc9-njt5f"
Jul 20 18:26:38.418630 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.418552    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk8vg\" (UniqueName: \"kubernetes.io/projected/980c1960-d168-43a0-8e4a-f673eda04cf0-kube-api-access-lk8vg\") pod \"maas-controller-7bf49cccc9-njt5f\" (UID: \"980c1960-d168-43a0-8e4a-f673eda04cf0\") " pod="opendatahub/maas-controller-7bf49cccc9-njt5f"
Jul 20 18:26:38.418778 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.418716    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/980c1960-d168-43a0-8e4a-f673eda04cf0-webhook-cert\") pod \"maas-controller-7bf49cccc9-njt5f\" (UID: \"980c1960-d168-43a0-8e4a-f673eda04cf0\") " pod="opendatahub/maas-controller-7bf49cccc9-njt5f"
Jul 20 18:26:38.421133 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.421111    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/980c1960-d168-43a0-8e4a-f673eda04cf0-webhook-cert\") pod \"maas-controller-7bf49cccc9-njt5f\" (UID: \"980c1960-d168-43a0-8e4a-f673eda04cf0\") " pod="opendatahub/maas-controller-7bf49cccc9-njt5f"
Jul 20 18:26:38.428906 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.428882    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk8vg\" (UniqueName: \"kubernetes.io/projected/980c1960-d168-43a0-8e4a-f673eda04cf0-kube-api-access-lk8vg\") pod \"maas-controller-7bf49cccc9-njt5f\" (UID: \"980c1960-d168-43a0-8e4a-f673eda04cf0\") " pod="opendatahub/maas-controller-7bf49cccc9-njt5f"
Jul 20 18:26:38.479751 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.479712    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bf49cccc9-njt5f"
Jul 20 18:26:38.616851 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.616826    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bf49cccc9-njt5f"]
Jul 20 18:26:38.618982 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:26:38.618957    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod980c1960_d168_43a0_8e4a_f673eda04cf0.slice/crio-50c5b02de25ad0c90f1464c40503553f865b13bc23affee2233b74400d0a2c5f WatchSource:0}: Error finding container 50c5b02de25ad0c90f1464c40503553f865b13bc23affee2233b74400d0a2c5f: Status 404 returned error can't find the container with id 50c5b02de25ad0c90f1464c40503553f865b13bc23affee2233b74400d0a2c5f
Jul 20 18:26:38.619998 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.619974    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f504946f-dd3f-4362-b9e0-064fa3fa6036-webhook-cert\") pod \"maas-controller-86d7bc6697-d87bw\" (UID: \"f504946f-dd3f-4362-b9e0-064fa3fa6036\") " pod="opendatahub/maas-controller-86d7bc6697-d87bw"
Jul 20 18:26:38.622348 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.622330    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f504946f-dd3f-4362-b9e0-064fa3fa6036-webhook-cert\") pod \"maas-controller-86d7bc6697-d87bw\" (UID: \"f504946f-dd3f-4362-b9e0-064fa3fa6036\") " pod="opendatahub/maas-controller-86d7bc6697-d87bw"
Jul 20 18:26:38.852171 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.852132    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d7bc6697-d87bw"
Jul 20 18:26:38.991704 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:38.991675    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86d7bc6697-d87bw"]
Jul 20 18:26:38.993270 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:26:38.993245    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf504946f_dd3f_4362_b9e0_064fa3fa6036.slice/crio-9067bd938b06112b0ddaf06b44bfba87430320541a6bb823d73c9f6e122a1ebf WatchSource:0}: Error finding container 9067bd938b06112b0ddaf06b44bfba87430320541a6bb823d73c9f6e122a1ebf: Status 404 returned error can't find the container with id 9067bd938b06112b0ddaf06b44bfba87430320541a6bb823d73c9f6e122a1ebf
Jul 20 18:26:39.052966 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:39.052928    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d7bc6697-d87bw" event={"ID":"f504946f-dd3f-4362-b9e0-064fa3fa6036","Type":"ContainerStarted","Data":"9067bd938b06112b0ddaf06b44bfba87430320541a6bb823d73c9f6e122a1ebf"}
Jul 20 18:26:39.054073 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:39.054029    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bf49cccc9-njt5f" event={"ID":"980c1960-d168-43a0-8e4a-f673eda04cf0","Type":"ContainerStarted","Data":"50c5b02de25ad0c90f1464c40503553f865b13bc23affee2233b74400d0a2c5f"}
Jul 20 18:26:41.238956 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:41.238922    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:26:41.337653 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:41.337525    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:26:42.070730 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:42.070689    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bf49cccc9-njt5f" event={"ID":"980c1960-d168-43a0-8e4a-f673eda04cf0","Type":"ContainerStarted","Data":"556c5f18f96065085ba35d3e00cbbaaa61537b10d9f5b29d9bb51e517d125179"}
Jul 20 18:26:42.070974 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:42.070856    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bf49cccc9-njt5f"
Jul 20 18:26:42.072212 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:42.072163    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d7bc6697-d87bw" event={"ID":"f504946f-dd3f-4362-b9e0-064fa3fa6036","Type":"ContainerStarted","Data":"2e7002cdd5932e6f27ce23de32238b2c2815ba790122b55b78c8084d53001fd8"}
Jul 20 18:26:42.072336 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:42.072224    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86d7bc6697-d87bw"
Jul 20 18:26:42.092254 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:42.092205    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7bf49cccc9-njt5f" podStartSLOduration=1.664168121 podStartE2EDuration="4.092187449s" totalImagesPullingTime="2.428019328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:26:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:26:42.090520593 +0000 UTC m=+798.924493007" watchObservedRunningTime="2026-07-20 18:26:42.092187449 +0000 UTC m=+798.926159870"
Jul 20 18:26:42.108852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:42.108802    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-86d7bc6697-d87bw" podStartSLOduration=3.065598219 podStartE2EDuration="5.108786801s" totalImagesPullingTime="2.043188582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:26:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:26:42.107814377 +0000 UTC m=+798.941786784" watchObservedRunningTime="2026-07-20 18:26:42.108786801 +0000 UTC m=+798.942759189"
Jul 20 18:26:53.079023 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:53.078992    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86d7bc6697-d87bw"
Jul 20 18:26:53.079508 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:53.079101    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bf49cccc9-njt5f"
Jul 20 18:26:53.145464 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:53.145434    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86d7bc6697-d87bw"]
Jul 20 18:26:53.145654 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:53.145602    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-86d7bc6697-d87bw" podUID="f504946f-dd3f-4362-b9e0-064fa3fa6036" containerName="manager" containerID="cri-o://2e7002cdd5932e6f27ce23de32238b2c2815ba790122b55b78c8084d53001fd8" gracePeriod=30
Jul 20 18:26:53.394765 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:53.394741    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d7bc6697-d87bw"
Jul 20 18:26:53.459934 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:53.459903    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9c6ws\" (UniqueName: \"kubernetes.io/projected/f504946f-dd3f-4362-b9e0-064fa3fa6036-kube-api-access-9c6ws\") pod \"f504946f-dd3f-4362-b9e0-064fa3fa6036\" (UID: \"f504946f-dd3f-4362-b9e0-064fa3fa6036\") "
Jul 20 18:26:53.460154 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:53.460021    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f504946f-dd3f-4362-b9e0-064fa3fa6036-webhook-cert\") pod \"f504946f-dd3f-4362-b9e0-064fa3fa6036\" (UID: \"f504946f-dd3f-4362-b9e0-064fa3fa6036\") "
Jul 20 18:26:53.462076 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:53.462039    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f504946f-dd3f-4362-b9e0-064fa3fa6036-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f504946f-dd3f-4362-b9e0-064fa3fa6036" (UID: "f504946f-dd3f-4362-b9e0-064fa3fa6036"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:26:53.462154 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:53.462134    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f504946f-dd3f-4362-b9e0-064fa3fa6036-kube-api-access-9c6ws" (OuterVolumeSpecName: "kube-api-access-9c6ws") pod "f504946f-dd3f-4362-b9e0-064fa3fa6036" (UID: "f504946f-dd3f-4362-b9e0-064fa3fa6036"). InnerVolumeSpecName "kube-api-access-9c6ws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:26:53.561696 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:53.561665    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f504946f-dd3f-4362-b9e0-064fa3fa6036-webhook-cert\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:26:53.561696 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:53.561692    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c6ws\" (UniqueName: \"kubernetes.io/projected/f504946f-dd3f-4362-b9e0-064fa3fa6036-kube-api-access-9c6ws\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:26:54.130582 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:54.130499    2569 generic.go:356] "Generic (PLEG): container finished" podID="f504946f-dd3f-4362-b9e0-064fa3fa6036" containerID="2e7002cdd5932e6f27ce23de32238b2c2815ba790122b55b78c8084d53001fd8" exitCode=0
Jul 20 18:26:54.130582 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:54.130537    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d7bc6697-d87bw"
Jul 20 18:26:54.131026 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:54.130579    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d7bc6697-d87bw" event={"ID":"f504946f-dd3f-4362-b9e0-064fa3fa6036","Type":"ContainerDied","Data":"2e7002cdd5932e6f27ce23de32238b2c2815ba790122b55b78c8084d53001fd8"}
Jul 20 18:26:54.131026 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:54.130627    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d7bc6697-d87bw" event={"ID":"f504946f-dd3f-4362-b9e0-064fa3fa6036","Type":"ContainerDied","Data":"9067bd938b06112b0ddaf06b44bfba87430320541a6bb823d73c9f6e122a1ebf"}
Jul 20 18:26:54.131026 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:54.130649    2569 scope.go:117] "RemoveContainer" containerID="2e7002cdd5932e6f27ce23de32238b2c2815ba790122b55b78c8084d53001fd8"
Jul 20 18:26:54.139252 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:54.139237    2569 scope.go:117] "RemoveContainer" containerID="2e7002cdd5932e6f27ce23de32238b2c2815ba790122b55b78c8084d53001fd8"
Jul 20 18:26:54.139501 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:26:54.139484    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e7002cdd5932e6f27ce23de32238b2c2815ba790122b55b78c8084d53001fd8\": container with ID starting with 2e7002cdd5932e6f27ce23de32238b2c2815ba790122b55b78c8084d53001fd8 not found: ID does not exist" containerID="2e7002cdd5932e6f27ce23de32238b2c2815ba790122b55b78c8084d53001fd8"
Jul 20 18:26:54.139549 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:54.139510    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e7002cdd5932e6f27ce23de32238b2c2815ba790122b55b78c8084d53001fd8"} err="failed to get container status \"2e7002cdd5932e6f27ce23de32238b2c2815ba790122b55b78c8084d53001fd8\": rpc error: code = NotFound desc = could not find container \"2e7002cdd5932e6f27ce23de32238b2c2815ba790122b55b78c8084d53001fd8\": container with ID starting with 2e7002cdd5932e6f27ce23de32238b2c2815ba790122b55b78c8084d53001fd8 not found: ID does not exist"
Jul 20 18:26:54.149692 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:54.149639    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86d7bc6697-d87bw"]
Jul 20 18:26:54.152001 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:54.151981    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-86d7bc6697-d87bw"]
Jul 20 18:26:55.759287 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:26:55.759248    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f504946f-dd3f-4362-b9e0-064fa3fa6036" path="/var/lib/kubelet/pods/f504946f-dd3f-4362-b9e0-064fa3fa6036/volumes"
Jul 20 18:27:12.371869 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:12.371835    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:27:12.488555 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:12.488512    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:27:12.880847 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:12.880813    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:27:12.930740 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:12.930702    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq"]
Jul 20 18:27:12.931326 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:12.931310    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f504946f-dd3f-4362-b9e0-064fa3fa6036" containerName="manager"
Jul 20 18:27:12.931326 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:12.931326    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f504946f-dd3f-4362-b9e0-064fa3fa6036" containerName="manager"
Jul 20 18:27:12.931481 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:12.931454    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f504946f-dd3f-4362-b9e0-064fa3fa6036" containerName="manager"
Jul 20 18:27:12.937895 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:12.937872    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq"
Jul 20 18:27:12.941014 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:12.940991    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 20 18:27:12.949119 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:12.949097    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq"]
Jul 20 18:27:13.036103 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.036046    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e05782-6e05-475e-bf83-c4d88fc37bce-openshift-service-ca\") pod \"maas-api-6bc7c4f9fb-99rgq\" (UID: \"b9e05782-6e05-475e-bf83-c4d88fc37bce\") " pod="odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq"
Jul 20 18:27:13.036264 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.036151    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9e05782-6e05-475e-bf83-c4d88fc37bce-maas-api-tls\") pod \"maas-api-6bc7c4f9fb-99rgq\" (UID: \"b9e05782-6e05-475e-bf83-c4d88fc37bce\") " pod="odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq"
Jul 20 18:27:13.036264 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.036219    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdtm4\" (UniqueName: \"kubernetes.io/projected/b9e05782-6e05-475e-bf83-c4d88fc37bce-kube-api-access-wdtm4\") pod \"maas-api-6bc7c4f9fb-99rgq\" (UID: \"b9e05782-6e05-475e-bf83-c4d88fc37bce\") " pod="odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq"
Jul 20 18:27:13.137488 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.137394    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdtm4\" (UniqueName: \"kubernetes.io/projected/b9e05782-6e05-475e-bf83-c4d88fc37bce-kube-api-access-wdtm4\") pod \"maas-api-6bc7c4f9fb-99rgq\" (UID: \"b9e05782-6e05-475e-bf83-c4d88fc37bce\") " pod="odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq"
Jul 20 18:27:13.137488 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.137463    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e05782-6e05-475e-bf83-c4d88fc37bce-openshift-service-ca\") pod \"maas-api-6bc7c4f9fb-99rgq\" (UID: \"b9e05782-6e05-475e-bf83-c4d88fc37bce\") " pod="odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq"
Jul 20 18:27:13.137733 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.137527    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9e05782-6e05-475e-bf83-c4d88fc37bce-maas-api-tls\") pod \"maas-api-6bc7c4f9fb-99rgq\" (UID: \"b9e05782-6e05-475e-bf83-c4d88fc37bce\") " pod="odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq"
Jul 20 18:27:13.138177 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.138156    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e05782-6e05-475e-bf83-c4d88fc37bce-openshift-service-ca\") pod \"maas-api-6bc7c4f9fb-99rgq\" (UID: \"b9e05782-6e05-475e-bf83-c4d88fc37bce\") " pod="odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq"
Jul 20 18:27:13.139940 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.139923    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9e05782-6e05-475e-bf83-c4d88fc37bce-maas-api-tls\") pod \"maas-api-6bc7c4f9fb-99rgq\" (UID: \"b9e05782-6e05-475e-bf83-c4d88fc37bce\") " pod="odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq"
Jul 20 18:27:13.151331 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.151306    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdtm4\" (UniqueName: \"kubernetes.io/projected/b9e05782-6e05-475e-bf83-c4d88fc37bce-kube-api-access-wdtm4\") pod \"maas-api-6bc7c4f9fb-99rgq\" (UID: \"b9e05782-6e05-475e-bf83-c4d88fc37bce\") " pod="odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq"
Jul 20 18:27:13.181739 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.181697    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:27:13.249402 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.249368    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq"
Jul 20 18:27:13.429071 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.429011    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq"]
Jul 20 18:27:13.431881 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:27:13.431853    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9e05782_6e05_475e_bf83_c4d88fc37bce.slice/crio-49f2c4ceec1ebea02a7c96a931240a8df3f9c94ee4bb52beae6246c3be0b899f WatchSource:0}: Error finding container 49f2c4ceec1ebea02a7c96a931240a8df3f9c94ee4bb52beae6246c3be0b899f: Status 404 returned error can't find the container with id 49f2c4ceec1ebea02a7c96a931240a8df3f9c94ee4bb52beae6246c3be0b899f
Jul 20 18:27:13.687244 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.687162    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:27:13.826232 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.826193    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-vxmcq"]
Jul 20 18:27:13.829861 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.829837    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-vxmcq"
Jul 20 18:27:13.832501 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.832474    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 20 18:27:13.855461 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.855432    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-vxmcq"]
Jul 20 18:27:13.857445 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.857412    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:27:13.944637 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.944528    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4v6x\" (UniqueName: \"kubernetes.io/projected/a93d5fef-3f3b-45e6-a416-2a13e1b2286d-kube-api-access-v4v6x\") pod \"payload-processing-5bf47cd988-vxmcq\" (UID: \"a93d5fef-3f3b-45e6-a416-2a13e1b2286d\") " pod="openshift-ingress/payload-processing-5bf47cd988-vxmcq"
Jul 20 18:27:13.944793 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:13.944684    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a93d5fef-3f3b-45e6-a416-2a13e1b2286d-plugins-config-volume\") pod \"payload-processing-5bf47cd988-vxmcq\" (UID: \"a93d5fef-3f3b-45e6-a416-2a13e1b2286d\") " pod="openshift-ingress/payload-processing-5bf47cd988-vxmcq"
Jul 20 18:27:14.045563 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:14.045517    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4v6x\" (UniqueName: \"kubernetes.io/projected/a93d5fef-3f3b-45e6-a416-2a13e1b2286d-kube-api-access-v4v6x\") pod \"payload-processing-5bf47cd988-vxmcq\" (UID: \"a93d5fef-3f3b-45e6-a416-2a13e1b2286d\") " pod="openshift-ingress/payload-processing-5bf47cd988-vxmcq"
Jul 20 18:27:14.045811 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:14.045636    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a93d5fef-3f3b-45e6-a416-2a13e1b2286d-plugins-config-volume\") pod \"payload-processing-5bf47cd988-vxmcq\" (UID: \"a93d5fef-3f3b-45e6-a416-2a13e1b2286d\") " pod="openshift-ingress/payload-processing-5bf47cd988-vxmcq"
Jul 20 18:27:14.046452 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:14.046421    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a93d5fef-3f3b-45e6-a416-2a13e1b2286d-plugins-config-volume\") pod \"payload-processing-5bf47cd988-vxmcq\" (UID: \"a93d5fef-3f3b-45e6-a416-2a13e1b2286d\") " pod="openshift-ingress/payload-processing-5bf47cd988-vxmcq"
Jul 20 18:27:14.097562 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:14.097533    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4v6x\" (UniqueName: \"kubernetes.io/projected/a93d5fef-3f3b-45e6-a416-2a13e1b2286d-kube-api-access-v4v6x\") pod \"payload-processing-5bf47cd988-vxmcq\" (UID: \"a93d5fef-3f3b-45e6-a416-2a13e1b2286d\") " pod="openshift-ingress/payload-processing-5bf47cd988-vxmcq"
Jul 20 18:27:14.142651 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:14.142621    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-vxmcq"
Jul 20 18:27:14.229948 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:14.229912    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq" event={"ID":"b9e05782-6e05-475e-bf83-c4d88fc37bce","Type":"ContainerStarted","Data":"49f2c4ceec1ebea02a7c96a931240a8df3f9c94ee4bb52beae6246c3be0b899f"}
Jul 20 18:27:14.354367 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:14.354340    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-vxmcq"]
Jul 20 18:27:14.356807 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:27:14.356779    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda93d5fef_3f3b_45e6_a416_2a13e1b2286d.slice/crio-bac4cca597905048a4da040ab793fb1e0ce3ec653b1ccfd4418e769cbd0d6196 WatchSource:0}: Error finding container bac4cca597905048a4da040ab793fb1e0ce3ec653b1ccfd4418e769cbd0d6196: Status 404 returned error can't find the container with id bac4cca597905048a4da040ab793fb1e0ce3ec653b1ccfd4418e769cbd0d6196
Jul 20 18:27:15.238071 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:15.237936    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-vxmcq" event={"ID":"a93d5fef-3f3b-45e6-a416-2a13e1b2286d","Type":"ContainerStarted","Data":"bac4cca597905048a4da040ab793fb1e0ce3ec653b1ccfd4418e769cbd0d6196"}
Jul 20 18:27:15.240250 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:15.240210    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq" event={"ID":"b9e05782-6e05-475e-bf83-c4d88fc37bce","Type":"ContainerStarted","Data":"ccba21f670dcd9679da740f057c4db25e9839a63a9b654011999adb16f905512"}
Jul 20 18:27:15.240393 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:15.240346    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq"
Jul 20 18:27:15.277408 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:15.277349    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq" podStartSLOduration=1.928729153 podStartE2EDuration="3.277319442s" totalImagesPullingTime="1.348590289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:27:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:27:15.276760261 +0000 UTC m=+832.110732650" watchObservedRunningTime="2026-07-20 18:27:15.277319442 +0000 UTC m=+832.111291831"
Jul 20 18:27:17.252093 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:17.252042    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-vxmcq" event={"ID":"a93d5fef-3f3b-45e6-a416-2a13e1b2286d","Type":"ContainerStarted","Data":"6282888d771c63bedbfed6f5242e9f5d26ba01e3482bead1bc010ff55c121c7a"}
Jul 20 18:27:17.252555 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:17.252207    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-vxmcq"
Jul 20 18:27:17.288985 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:17.288931    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-vxmcq" podStartSLOduration=1.904557755 podStartE2EDuration="4.288916033s" totalImagesPullingTime="2.384358278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:27:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:27:17.284226865 +0000 UTC m=+834.118199277" watchObservedRunningTime="2026-07-20 18:27:17.288916033 +0000 UTC m=+834.122888421"
Jul 20 18:27:21.251015 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:21.250989    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6bc7c4f9fb-99rgq"
Jul 20 18:27:22.307628 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:22.307585    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:27:22.417163 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:22.417123    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:27:28.259248 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:28.259208    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-vxmcq"
Jul 20 18:27:33.692834 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:33.692796    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:27:33.792835 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:33.792799    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:27:37.849457 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:37.849421    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:27:38.114140 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:38.114040    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:27:53.820187 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:53.820155    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"]
Jul 20 18:27:53.822741 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:53.822716    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:53.827695 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:53.827667    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-79fz4\""
Jul 20 18:27:53.830327 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:53.830306    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 20 18:27:53.835039 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:53.835021    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 20 18:27:53.836699 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:53.836680    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 20 18:27:53.850997 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:53.850966    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"]
Jul 20 18:27:53.883937 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:53.883906    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:27:53.906905 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:53.906872    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3adbfffe-6de8-42a2-9379-ceda509de514-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:53.907085 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:53.906915    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3adbfffe-6de8-42a2-9379-ceda509de514-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:53.907085 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:53.906937    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qgtl\" (UniqueName: \"kubernetes.io/projected/3adbfffe-6de8-42a2-9379-ceda509de514-kube-api-access-6qgtl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:53.907085 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:53.907008    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3adbfffe-6de8-42a2-9379-ceda509de514-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:53.907085 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:53.907077    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3adbfffe-6de8-42a2-9379-ceda509de514-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:53.907236 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:53.907100    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3adbfffe-6de8-42a2-9379-ceda509de514-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:54.008634 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:54.008594    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3adbfffe-6de8-42a2-9379-ceda509de514-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:54.008634 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:54.008638    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3adbfffe-6de8-42a2-9379-ceda509de514-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:54.008869 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:54.008722    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3adbfffe-6de8-42a2-9379-ceda509de514-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:54.008869 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:54.008760    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3adbfffe-6de8-42a2-9379-ceda509de514-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:54.008869 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:54.008782    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qgtl\" (UniqueName: \"kubernetes.io/projected/3adbfffe-6de8-42a2-9379-ceda509de514-kube-api-access-6qgtl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:54.008869 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:54.008812    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3adbfffe-6de8-42a2-9379-ceda509de514-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:54.009152 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:54.009115    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3adbfffe-6de8-42a2-9379-ceda509de514-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:54.009152 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:54.009131    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3adbfffe-6de8-42a2-9379-ceda509de514-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:54.009260 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:54.009181    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3adbfffe-6de8-42a2-9379-ceda509de514-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:54.010816 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:54.010798    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3adbfffe-6de8-42a2-9379-ceda509de514-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:54.010982 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:54.010965    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3adbfffe-6de8-42a2-9379-ceda509de514-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:54.018928 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:54.018906    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qgtl\" (UniqueName: \"kubernetes.io/projected/3adbfffe-6de8-42a2-9379-ceda509de514-kube-api-access-6qgtl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn\" (UID: \"3adbfffe-6de8-42a2-9379-ceda509de514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:54.134041 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:54.133956    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:27:54.276694 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:54.276665    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"]
Jul 20 18:27:54.279581 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:27:54.279541    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3adbfffe_6de8_42a2_9379_ceda509de514.slice/crio-552a35c5f5f0ac4c01b2818d2aaf02db0fb37de4c7d9b115f82443d80002c27a WatchSource:0}: Error finding container 552a35c5f5f0ac4c01b2818d2aaf02db0fb37de4c7d9b115f82443d80002c27a: Status 404 returned error can't find the container with id 552a35c5f5f0ac4c01b2818d2aaf02db0fb37de4c7d9b115f82443d80002c27a
Jul 20 18:27:54.423084 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:54.422983    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn" event={"ID":"3adbfffe-6de8-42a2-9379-ceda509de514","Type":"ContainerStarted","Data":"552a35c5f5f0ac4c01b2818d2aaf02db0fb37de4c7d9b115f82443d80002c27a"}
Jul 20 18:27:56.435591 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:27:56.435556    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn" event={"ID":"3adbfffe-6de8-42a2-9379-ceda509de514","Type":"ContainerStarted","Data":"d3f164dea66338f50b8befd889b275e5cdb7223ea17681d95308482a8d42fb52"}
Jul 20 18:28:04.134815 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:04.134778    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:28:04.135227 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:04.134841    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:28:04.147573 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:04.147548    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:28:04.168431 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:04.168378    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn" podStartSLOduration=10.037168572 podStartE2EDuration="11.168363493s" totalImagesPullingTime="1.131194921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:27:56.456998972 +0000 UTC m=+873.290971361" watchObservedRunningTime="2026-07-20 18:28:04.168363493 +0000 UTC m=+881.002335881"
Jul 20 18:28:04.483763 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:04.483679    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn"
Jul 20 18:28:05.197383 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:05.197349    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:28:17.385584 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:17.385507    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:28:30.148281 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:30.148244    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:28:30.149722 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:30.149698    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:28:30.152782 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:30.152763    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:28:52.509391 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.509360    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"]
Jul 20 18:28:52.511859 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.511842    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.515064 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.515037    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 20 18:28:52.526169 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.526147    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"]
Jul 20 18:28:52.649737 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.649700    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f09038b-3163-4930-b937-68ee9290ab19-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.649737 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.649742    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f09038b-3163-4930-b937-68ee9290ab19-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.649936 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.649759    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f09038b-3163-4930-b937-68ee9290ab19-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.649936 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.649834    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f09038b-3163-4930-b937-68ee9290ab19-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.649936 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.649920    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptm66\" (UniqueName: \"kubernetes.io/projected/6f09038b-3163-4930-b937-68ee9290ab19-kube-api-access-ptm66\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.650038 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.649988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f09038b-3163-4930-b937-68ee9290ab19-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.683790 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.683759    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:28:52.750765 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.750734    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f09038b-3163-4930-b937-68ee9290ab19-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.750765 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.750770    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f09038b-3163-4930-b937-68ee9290ab19-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.751026 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.750789    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f09038b-3163-4930-b937-68ee9290ab19-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.751026 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.750915    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f09038b-3163-4930-b937-68ee9290ab19-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.751026 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.751003    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptm66\" (UniqueName: \"kubernetes.io/projected/6f09038b-3163-4930-b937-68ee9290ab19-kube-api-access-ptm66\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.751212 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.751152    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f09038b-3163-4930-b937-68ee9290ab19-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.751212 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.751189    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f09038b-3163-4930-b937-68ee9290ab19-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.751316 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.751232    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f09038b-3163-4930-b937-68ee9290ab19-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.751316 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.751275    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f09038b-3163-4930-b937-68ee9290ab19-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.753017 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.752990    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f09038b-3163-4930-b937-68ee9290ab19-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.753399 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.753380    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f09038b-3163-4930-b937-68ee9290ab19-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.769236 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.769173    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptm66\" (UniqueName: \"kubernetes.io/projected/6f09038b-3163-4930-b937-68ee9290ab19-kube-api-access-ptm66\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-czd87\" (UID: \"6f09038b-3163-4930-b937-68ee9290ab19\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.821668 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.821627    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:28:52.955531 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:52.955509    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"]
Jul 20 18:28:52.957676 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:28:52.957650    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f09038b_3163_4930_b937_68ee9290ab19.slice/crio-f7bfbb88540b761d853867cfb6ea2e462aab94f76320261026e39f83e6aa209d WatchSource:0}: Error finding container f7bfbb88540b761d853867cfb6ea2e462aab94f76320261026e39f83e6aa209d: Status 404 returned error can't find the container with id f7bfbb88540b761d853867cfb6ea2e462aab94f76320261026e39f83e6aa209d
Jul 20 18:28:53.709891 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:53.709858    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87" event={"ID":"6f09038b-3163-4930-b937-68ee9290ab19","Type":"ContainerStarted","Data":"600dad4d0761a7482a765de09ecebebd4fe19114e4c10db6a4236dd844e158d9"}
Jul 20 18:28:53.710313 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:53.709915    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87" event={"ID":"6f09038b-3163-4930-b937-68ee9290ab19","Type":"ContainerStarted","Data":"f7bfbb88540b761d853867cfb6ea2e462aab94f76320261026e39f83e6aa209d"}
Jul 20 18:28:53.731599 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:28:53.731544    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87" podStartSLOduration=1.547071938 podStartE2EDuration="1.731528445s" totalImagesPullingTime="184.456507ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:28:53.728902385 +0000 UTC m=+930.562874774" watchObservedRunningTime="2026-07-20 18:28:53.731528445 +0000 UTC m=+930.565500833"
Jul 20 18:29:02.822589 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:29:02.822553    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:29:02.823228 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:29:02.822621    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:29:02.835766 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:29:02.835739    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:29:03.769988 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:29:03.769960    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-czd87"
Jul 20 18:29:11.458323 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:29:11.458289    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:29:44.512712 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:29:44.512675    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:29:44.573700 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:29:44.573659    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:30:00.145982 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:00.145947    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742870-26t84"]
Jul 20 18:30:00.149395 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:00.149377    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742870-26t84"
Jul 20 18:30:00.152280 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:00.152255    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w8qkh\""
Jul 20 18:30:00.154646 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:00.154622    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742870-26t84"]
Jul 20 18:30:00.161086 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:00.161046    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742870-26t84"
Jul 20 18:30:00.298027 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:00.298004    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742870-26t84"]
Jul 20 18:30:00.300810 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:30:00.300778    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1ce75de_77ee_4866_a2b7_dd728e4f351a.slice/crio-5f449e3b219b6f82ab0b049f5a4542c1c5f1b783ac78754653c48f0a8b154f6e WatchSource:0}: Error finding container 5f449e3b219b6f82ab0b049f5a4542c1c5f1b783ac78754653c48f0a8b154f6e: Status 404 returned error can't find the container with id 5f449e3b219b6f82ab0b049f5a4542c1c5f1b783ac78754653c48f0a8b154f6e
Jul 20 18:30:01.032836 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:01.032800    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742870-26t84" event={"ID":"e1ce75de-77ee-4866-a2b7-dd728e4f351a","Type":"ContainerStarted","Data":"5f449e3b219b6f82ab0b049f5a4542c1c5f1b783ac78754653c48f0a8b154f6e"}
Jul 20 18:30:03.044326 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:03.044283    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742870-26t84" event={"ID":"e1ce75de-77ee-4866-a2b7-dd728e4f351a","Type":"ContainerStarted","Data":"172f1047da916ddf1cf69eae41fb704895b92ad77982263aff8dd1c4686f13ff"}
Jul 20 18:30:03.061944 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:03.061886    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742870-26t84" podStartSLOduration=0.674878565 podStartE2EDuration="3.061870003s" totalImagesPullingTime="2.386991438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:30:03.060011699 +0000 UTC m=+999.893984088" watchObservedRunningTime="2026-07-20 18:30:03.061870003 +0000 UTC m=+999.895842391"
Jul 20 18:30:04.051003 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:04.050968    2569 generic.go:356] "Generic (PLEG): container finished" podID="e1ce75de-77ee-4866-a2b7-dd728e4f351a" containerID="172f1047da916ddf1cf69eae41fb704895b92ad77982263aff8dd1c4686f13ff" exitCode=0
Jul 20 18:30:04.051445 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:04.051007    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742870-26t84" event={"ID":"e1ce75de-77ee-4866-a2b7-dd728e4f351a","Type":"ContainerDied","Data":"172f1047da916ddf1cf69eae41fb704895b92ad77982263aff8dd1c4686f13ff"}
Jul 20 18:30:05.190349 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:05.190322    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742870-26t84"
Jul 20 18:30:06.063301 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:06.063213    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742870-26t84" event={"ID":"e1ce75de-77ee-4866-a2b7-dd728e4f351a","Type":"ContainerDied","Data":"5f449e3b219b6f82ab0b049f5a4542c1c5f1b783ac78754653c48f0a8b154f6e"}
Jul 20 18:30:06.063301 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:06.063280    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f449e3b219b6f82ab0b049f5a4542c1c5f1b783ac78754653c48f0a8b154f6e"
Jul 20 18:30:06.063301 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:06.063223    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742870-26t84"
Jul 20 18:30:24.684241 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:24.684214    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:30:24.781949 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:24.781906    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:30:25.843646 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:25.843609    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:30:25.932162 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:30:25.932124    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:31:11.946702 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:11.946666    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bf49cccc9-njt5f"]
Jul 20 18:31:11.947242 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:11.946915    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7bf49cccc9-njt5f" podUID="980c1960-d168-43a0-8e4a-f673eda04cf0" containerName="manager" containerID="cri-o://556c5f18f96065085ba35d3e00cbbaaa61537b10d9f5b29d9bb51e517d125179" gracePeriod=30
Jul 20 18:31:12.189959 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:12.189932    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bf49cccc9-njt5f"
Jul 20 18:31:12.222228 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:12.222157    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lk8vg\" (UniqueName: \"kubernetes.io/projected/980c1960-d168-43a0-8e4a-f673eda04cf0-kube-api-access-lk8vg\") pod \"980c1960-d168-43a0-8e4a-f673eda04cf0\" (UID: \"980c1960-d168-43a0-8e4a-f673eda04cf0\") "
Jul 20 18:31:12.222356 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:12.222259    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/980c1960-d168-43a0-8e4a-f673eda04cf0-webhook-cert\") pod \"980c1960-d168-43a0-8e4a-f673eda04cf0\" (UID: \"980c1960-d168-43a0-8e4a-f673eda04cf0\") "
Jul 20 18:31:12.224350 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:12.224319    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/980c1960-d168-43a0-8e4a-f673eda04cf0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "980c1960-d168-43a0-8e4a-f673eda04cf0" (UID: "980c1960-d168-43a0-8e4a-f673eda04cf0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:31:12.224350 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:12.224324    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/980c1960-d168-43a0-8e4a-f673eda04cf0-kube-api-access-lk8vg" (OuterVolumeSpecName: "kube-api-access-lk8vg") pod "980c1960-d168-43a0-8e4a-f673eda04cf0" (UID: "980c1960-d168-43a0-8e4a-f673eda04cf0"). InnerVolumeSpecName "kube-api-access-lk8vg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:31:12.323037 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:12.323005    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lk8vg\" (UniqueName: \"kubernetes.io/projected/980c1960-d168-43a0-8e4a-f673eda04cf0-kube-api-access-lk8vg\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:31:12.323037 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:12.323032    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/980c1960-d168-43a0-8e4a-f673eda04cf0-webhook-cert\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:31:12.398163 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:12.398127    2569 generic.go:356] "Generic (PLEG): container finished" podID="980c1960-d168-43a0-8e4a-f673eda04cf0" containerID="556c5f18f96065085ba35d3e00cbbaaa61537b10d9f5b29d9bb51e517d125179" exitCode=0
Jul 20 18:31:12.398352 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:12.398165    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bf49cccc9-njt5f" event={"ID":"980c1960-d168-43a0-8e4a-f673eda04cf0","Type":"ContainerDied","Data":"556c5f18f96065085ba35d3e00cbbaaa61537b10d9f5b29d9bb51e517d125179"}
Jul 20 18:31:12.398352 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:12.398200    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bf49cccc9-njt5f"
Jul 20 18:31:12.398352 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:12.398235    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bf49cccc9-njt5f" event={"ID":"980c1960-d168-43a0-8e4a-f673eda04cf0","Type":"ContainerDied","Data":"50c5b02de25ad0c90f1464c40503553f865b13bc23affee2233b74400d0a2c5f"}
Jul 20 18:31:12.398352 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:12.398266    2569 scope.go:117] "RemoveContainer" containerID="556c5f18f96065085ba35d3e00cbbaaa61537b10d9f5b29d9bb51e517d125179"
Jul 20 18:31:12.407813 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:12.407795    2569 scope.go:117] "RemoveContainer" containerID="556c5f18f96065085ba35d3e00cbbaaa61537b10d9f5b29d9bb51e517d125179"
Jul 20 18:31:12.408088 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:31:12.408070    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"556c5f18f96065085ba35d3e00cbbaaa61537b10d9f5b29d9bb51e517d125179\": container with ID starting with 556c5f18f96065085ba35d3e00cbbaaa61537b10d9f5b29d9bb51e517d125179 not found: ID does not exist" containerID="556c5f18f96065085ba35d3e00cbbaaa61537b10d9f5b29d9bb51e517d125179"
Jul 20 18:31:12.408145 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:12.408099    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"556c5f18f96065085ba35d3e00cbbaaa61537b10d9f5b29d9bb51e517d125179"} err="failed to get container status \"556c5f18f96065085ba35d3e00cbbaaa61537b10d9f5b29d9bb51e517d125179\": rpc error: code = NotFound desc = could not find container \"556c5f18f96065085ba35d3e00cbbaaa61537b10d9f5b29d9bb51e517d125179\": container with ID starting with 556c5f18f96065085ba35d3e00cbbaaa61537b10d9f5b29d9bb51e517d125179 not found: ID does not exist"
Jul 20 18:31:12.419767 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:12.419745    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bf49cccc9-njt5f"]
Jul 20 18:31:12.426229 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:12.426207    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7bf49cccc9-njt5f"]
Jul 20 18:31:13.759984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:13.759951    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="980c1960-d168-43a0-8e4a-f673eda04cf0" path="/var/lib/kubelet/pods/980c1960-d168-43a0-8e4a-f673eda04cf0/volumes"
Jul 20 18:31:23.667994 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:23.667915    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:31:40.921947 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:40.921914    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:31:49.014485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:49.014454    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:31:49.113123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:31:49.113038    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:32:13.246048 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.246013    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bf49cccc9-gqb2z"]
Jul 20 18:32:13.246533 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.246406    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="980c1960-d168-43a0-8e4a-f673eda04cf0" containerName="manager"
Jul 20 18:32:13.246533 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.246418    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="980c1960-d168-43a0-8e4a-f673eda04cf0" containerName="manager"
Jul 20 18:32:13.246533 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.246429    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1ce75de-77ee-4866-a2b7-dd728e4f351a" containerName="cleanup"
Jul 20 18:32:13.246533 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.246434    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1ce75de-77ee-4866-a2b7-dd728e4f351a" containerName="cleanup"
Jul 20 18:32:13.246533 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.246508    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1ce75de-77ee-4866-a2b7-dd728e4f351a" containerName="cleanup"
Jul 20 18:32:13.246533 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.246517    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="980c1960-d168-43a0-8e4a-f673eda04cf0" containerName="manager"
Jul 20 18:32:13.249511 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.249494    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bf49cccc9-gqb2z"
Jul 20 18:32:13.252811 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.252785    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-94fp5\""
Jul 20 18:32:13.252963 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.252821    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 20 18:32:13.252963 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.252845    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 20 18:32:13.261019 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.260996    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bf49cccc9-gqb2z"]
Jul 20 18:32:13.381710 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.381678    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77lpk\" (UniqueName: \"kubernetes.io/projected/9046575b-46f4-4ba4-97d5-5d3a8afbe8a6-kube-api-access-77lpk\") pod \"maas-controller-7bf49cccc9-gqb2z\" (UID: \"9046575b-46f4-4ba4-97d5-5d3a8afbe8a6\") " pod="opendatahub/maas-controller-7bf49cccc9-gqb2z"
Jul 20 18:32:13.381902 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.381729    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9046575b-46f4-4ba4-97d5-5d3a8afbe8a6-webhook-cert\") pod \"maas-controller-7bf49cccc9-gqb2z\" (UID: \"9046575b-46f4-4ba4-97d5-5d3a8afbe8a6\") " pod="opendatahub/maas-controller-7bf49cccc9-gqb2z"
Jul 20 18:32:13.482235 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.482199    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77lpk\" (UniqueName: \"kubernetes.io/projected/9046575b-46f4-4ba4-97d5-5d3a8afbe8a6-kube-api-access-77lpk\") pod \"maas-controller-7bf49cccc9-gqb2z\" (UID: \"9046575b-46f4-4ba4-97d5-5d3a8afbe8a6\") " pod="opendatahub/maas-controller-7bf49cccc9-gqb2z"
Jul 20 18:32:13.482428 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.482256    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9046575b-46f4-4ba4-97d5-5d3a8afbe8a6-webhook-cert\") pod \"maas-controller-7bf49cccc9-gqb2z\" (UID: \"9046575b-46f4-4ba4-97d5-5d3a8afbe8a6\") " pod="opendatahub/maas-controller-7bf49cccc9-gqb2z"
Jul 20 18:32:13.484671 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.484638    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9046575b-46f4-4ba4-97d5-5d3a8afbe8a6-webhook-cert\") pod \"maas-controller-7bf49cccc9-gqb2z\" (UID: \"9046575b-46f4-4ba4-97d5-5d3a8afbe8a6\") " pod="opendatahub/maas-controller-7bf49cccc9-gqb2z"
Jul 20 18:32:13.491553 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.491522    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77lpk\" (UniqueName: \"kubernetes.io/projected/9046575b-46f4-4ba4-97d5-5d3a8afbe8a6-kube-api-access-77lpk\") pod \"maas-controller-7bf49cccc9-gqb2z\" (UID: \"9046575b-46f4-4ba4-97d5-5d3a8afbe8a6\") " pod="opendatahub/maas-controller-7bf49cccc9-gqb2z"
Jul 20 18:32:13.561748 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.561644    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bf49cccc9-gqb2z"
Jul 20 18:32:13.698985 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.698954    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bf49cccc9-gqb2z"]
Jul 20 18:32:13.700517 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:32:13.700490    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9046575b_46f4_4ba4_97d5_5d3a8afbe8a6.slice/crio-f46db165f787208216ad8bfae08526d67cbe7e43e3edd1f9b80950aba1ebb753 WatchSource:0}: Error finding container f46db165f787208216ad8bfae08526d67cbe7e43e3edd1f9b80950aba1ebb753: Status 404 returned error can't find the container with id f46db165f787208216ad8bfae08526d67cbe7e43e3edd1f9b80950aba1ebb753
Jul 20 18:32:13.701774 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:13.701758    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 18:32:14.699278 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:14.699191    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bf49cccc9-gqb2z" event={"ID":"9046575b-46f4-4ba4-97d5-5d3a8afbe8a6","Type":"ContainerStarted","Data":"51d961b43f02090109a21cff9bacf049ab798dff8c9dbe6183e14efe28ee2611"}
Jul 20 18:32:14.699278 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:14.699247    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bf49cccc9-gqb2z" event={"ID":"9046575b-46f4-4ba4-97d5-5d3a8afbe8a6","Type":"ContainerStarted","Data":"f46db165f787208216ad8bfae08526d67cbe7e43e3edd1f9b80950aba1ebb753"}
Jul 20 18:32:14.699772 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:14.699322    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bf49cccc9-gqb2z"
Jul 20 18:32:14.719477 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:14.719427    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7bf49cccc9-gqb2z" podStartSLOduration=1.158832465 podStartE2EDuration="1.719412957s" totalImagesPullingTime="560.580492ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:32:14.717351281 +0000 UTC m=+1131.551323673" watchObservedRunningTime="2026-07-20 18:32:14.719412957 +0000 UTC m=+1131.553385345"
Jul 20 18:32:25.706412 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:25.706380    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bf49cccc9-gqb2z"
Jul 20 18:32:30.279313 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.279275    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:32:30.359334 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.359305    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bf49cccc9-gqb2z"]
Jul 20 18:32:30.359518 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.359495    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7bf49cccc9-gqb2z" podUID="9046575b-46f4-4ba4-97d5-5d3a8afbe8a6" containerName="manager" containerID="cri-o://51d961b43f02090109a21cff9bacf049ab798dff8c9dbe6183e14efe28ee2611" gracePeriod=30
Jul 20 18:32:30.596679 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.596656    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bf49cccc9-gqb2z"
Jul 20 18:32:30.637783 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.637754    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77lpk\" (UniqueName: \"kubernetes.io/projected/9046575b-46f4-4ba4-97d5-5d3a8afbe8a6-kube-api-access-77lpk\") pod \"9046575b-46f4-4ba4-97d5-5d3a8afbe8a6\" (UID: \"9046575b-46f4-4ba4-97d5-5d3a8afbe8a6\") "
Jul 20 18:32:30.637963 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.637811    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9046575b-46f4-4ba4-97d5-5d3a8afbe8a6-webhook-cert\") pod \"9046575b-46f4-4ba4-97d5-5d3a8afbe8a6\" (UID: \"9046575b-46f4-4ba4-97d5-5d3a8afbe8a6\") "
Jul 20 18:32:30.639910 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.639883    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9046575b-46f4-4ba4-97d5-5d3a8afbe8a6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9046575b-46f4-4ba4-97d5-5d3a8afbe8a6" (UID: "9046575b-46f4-4ba4-97d5-5d3a8afbe8a6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:32:30.640024 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.640002    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9046575b-46f4-4ba4-97d5-5d3a8afbe8a6-kube-api-access-77lpk" (OuterVolumeSpecName: "kube-api-access-77lpk") pod "9046575b-46f4-4ba4-97d5-5d3a8afbe8a6" (UID: "9046575b-46f4-4ba4-97d5-5d3a8afbe8a6"). InnerVolumeSpecName "kube-api-access-77lpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:32:30.738718 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.738666    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77lpk\" (UniqueName: \"kubernetes.io/projected/9046575b-46f4-4ba4-97d5-5d3a8afbe8a6-kube-api-access-77lpk\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:32:30.738718 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.738709    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9046575b-46f4-4ba4-97d5-5d3a8afbe8a6-webhook-cert\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:32:30.776878 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.776848    2569 generic.go:356] "Generic (PLEG): container finished" podID="9046575b-46f4-4ba4-97d5-5d3a8afbe8a6" containerID="51d961b43f02090109a21cff9bacf049ab798dff8c9dbe6183e14efe28ee2611" exitCode=0
Jul 20 18:32:30.777099 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.776897    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bf49cccc9-gqb2z"
Jul 20 18:32:30.777099 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.776938    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bf49cccc9-gqb2z" event={"ID":"9046575b-46f4-4ba4-97d5-5d3a8afbe8a6","Type":"ContainerDied","Data":"51d961b43f02090109a21cff9bacf049ab798dff8c9dbe6183e14efe28ee2611"}
Jul 20 18:32:30.777099 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.776993    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bf49cccc9-gqb2z" event={"ID":"9046575b-46f4-4ba4-97d5-5d3a8afbe8a6","Type":"ContainerDied","Data":"f46db165f787208216ad8bfae08526d67cbe7e43e3edd1f9b80950aba1ebb753"}
Jul 20 18:32:30.777099 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.777022    2569 scope.go:117] "RemoveContainer" containerID="51d961b43f02090109a21cff9bacf049ab798dff8c9dbe6183e14efe28ee2611"
Jul 20 18:32:30.786826 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.786807    2569 scope.go:117] "RemoveContainer" containerID="51d961b43f02090109a21cff9bacf049ab798dff8c9dbe6183e14efe28ee2611"
Jul 20 18:32:30.787106 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:32:30.787085    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51d961b43f02090109a21cff9bacf049ab798dff8c9dbe6183e14efe28ee2611\": container with ID starting with 51d961b43f02090109a21cff9bacf049ab798dff8c9dbe6183e14efe28ee2611 not found: ID does not exist" containerID="51d961b43f02090109a21cff9bacf049ab798dff8c9dbe6183e14efe28ee2611"
Jul 20 18:32:30.787162 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.787119    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51d961b43f02090109a21cff9bacf049ab798dff8c9dbe6183e14efe28ee2611"} err="failed to get container status \"51d961b43f02090109a21cff9bacf049ab798dff8c9dbe6183e14efe28ee2611\": rpc error: code = NotFound desc = could not find container \"51d961b43f02090109a21cff9bacf049ab798dff8c9dbe6183e14efe28ee2611\": container with ID starting with 51d961b43f02090109a21cff9bacf049ab798dff8c9dbe6183e14efe28ee2611 not found: ID does not exist"
Jul 20 18:32:30.798534 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.798512    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bf49cccc9-gqb2z"]
Jul 20 18:32:30.805524 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:30.805503    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7bf49cccc9-gqb2z"]
Jul 20 18:32:31.627590 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.627549    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bf49cccc9-dnbs7"]
Jul 20 18:32:31.628093 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.627946    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9046575b-46f4-4ba4-97d5-5d3a8afbe8a6" containerName="manager"
Jul 20 18:32:31.628093 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.627956    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9046575b-46f4-4ba4-97d5-5d3a8afbe8a6" containerName="manager"
Jul 20 18:32:31.628093 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.628042    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9046575b-46f4-4ba4-97d5-5d3a8afbe8a6" containerName="manager"
Jul 20 18:32:31.632297 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.632272    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bf49cccc9-dnbs7"
Jul 20 18:32:31.636207 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.636183    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 20 18:32:31.636977 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.636954    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-94fp5\""
Jul 20 18:32:31.637103 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.636975    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 20 18:32:31.648320 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.648293    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzh9l\" (UniqueName: \"kubernetes.io/projected/524e6c5a-090f-4d93-bd64-6564d2c72141-kube-api-access-pzh9l\") pod \"maas-controller-7bf49cccc9-dnbs7\" (UID: \"524e6c5a-090f-4d93-bd64-6564d2c72141\") " pod="opendatahub/maas-controller-7bf49cccc9-dnbs7"
Jul 20 18:32:31.648448 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.648428    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/524e6c5a-090f-4d93-bd64-6564d2c72141-webhook-cert\") pod \"maas-controller-7bf49cccc9-dnbs7\" (UID: \"524e6c5a-090f-4d93-bd64-6564d2c72141\") " pod="opendatahub/maas-controller-7bf49cccc9-dnbs7"
Jul 20 18:32:31.648542 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.648521    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bf49cccc9-dnbs7"]
Jul 20 18:32:31.749232 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.749200    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzh9l\" (UniqueName: \"kubernetes.io/projected/524e6c5a-090f-4d93-bd64-6564d2c72141-kube-api-access-pzh9l\") pod \"maas-controller-7bf49cccc9-dnbs7\" (UID: \"524e6c5a-090f-4d93-bd64-6564d2c72141\") " pod="opendatahub/maas-controller-7bf49cccc9-dnbs7"
Jul 20 18:32:31.749422 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.749297    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/524e6c5a-090f-4d93-bd64-6564d2c72141-webhook-cert\") pod \"maas-controller-7bf49cccc9-dnbs7\" (UID: \"524e6c5a-090f-4d93-bd64-6564d2c72141\") " pod="opendatahub/maas-controller-7bf49cccc9-dnbs7"
Jul 20 18:32:31.751665 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.751645    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/524e6c5a-090f-4d93-bd64-6564d2c72141-webhook-cert\") pod \"maas-controller-7bf49cccc9-dnbs7\" (UID: \"524e6c5a-090f-4d93-bd64-6564d2c72141\") " pod="opendatahub/maas-controller-7bf49cccc9-dnbs7"
Jul 20 18:32:31.758680 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.758655    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9046575b-46f4-4ba4-97d5-5d3a8afbe8a6" path="/var/lib/kubelet/pods/9046575b-46f4-4ba4-97d5-5d3a8afbe8a6/volumes"
Jul 20 18:32:31.759960 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.759937    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzh9l\" (UniqueName: \"kubernetes.io/projected/524e6c5a-090f-4d93-bd64-6564d2c72141-kube-api-access-pzh9l\") pod \"maas-controller-7bf49cccc9-dnbs7\" (UID: \"524e6c5a-090f-4d93-bd64-6564d2c72141\") " pod="opendatahub/maas-controller-7bf49cccc9-dnbs7"
Jul 20 18:32:31.943202 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:31.943113    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bf49cccc9-dnbs7"
Jul 20 18:32:32.082686 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:32.082662    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bf49cccc9-dnbs7"]
Jul 20 18:32:32.084697 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:32:32.084669    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod524e6c5a_090f_4d93_bd64_6564d2c72141.slice/crio-73a4362b4b472b8b2872a003a23f6083a860eefdd931e10c4e323c62616c098a WatchSource:0}: Error finding container 73a4362b4b472b8b2872a003a23f6083a860eefdd931e10c4e323c62616c098a: Status 404 returned error can't find the container with id 73a4362b4b472b8b2872a003a23f6083a860eefdd931e10c4e323c62616c098a
Jul 20 18:32:32.098132 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:32.098106    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:32:32.789146 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:32.789103    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bf49cccc9-dnbs7" event={"ID":"524e6c5a-090f-4d93-bd64-6564d2c72141","Type":"ContainerStarted","Data":"2a9da6c78ebce7c229196167a8b8b8aca5aa080ed1c4465211614e75cb13be73"}
Jul 20 18:32:32.789146 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:32.789152    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bf49cccc9-dnbs7" event={"ID":"524e6c5a-090f-4d93-bd64-6564d2c72141","Type":"ContainerStarted","Data":"73a4362b4b472b8b2872a003a23f6083a860eefdd931e10c4e323c62616c098a"}
Jul 20 18:32:32.789662 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:32.789247    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bf49cccc9-dnbs7"
Jul 20 18:32:32.812480 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:32.812371    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7bf49cccc9-dnbs7" podStartSLOduration=1.352631348 podStartE2EDuration="1.812356969s" totalImagesPullingTime="459.725621ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:32:32.811198487 +0000 UTC m=+1149.645170886" watchObservedRunningTime="2026-07-20 18:32:32.812356969 +0000 UTC m=+1149.646329357"
Jul 20 18:32:41.730519 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:41.730484    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:32:41.830528 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:41.830498    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:32:43.796275 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:32:43.796245    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bf49cccc9-dnbs7"
Jul 20 18:33:10.253701 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:33:10.253668    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:33:12.240825 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:33:12.240784    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:33:16.521439 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:33:16.521404    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:33:16.649621 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:33:16.649590    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:33:16.804699 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:33:16.804617    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:33:23.107684 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:33:23.107646    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:33:23.190406 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:33:23.190370    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:33:30.183231 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:33:30.183204    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:33:30.187223 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:33:30.187201    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:34:18.749415 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:34:18.749326    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:35:15.696396 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:35:15.696360    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:35:15.761914 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:35:15.761863    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:37:49.602362 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:37:49.602318    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:38:30.222481 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:38:30.222450    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:38:30.228719 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:38:30.228698    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:38:45.108863 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:38:45.108778    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:39:45.063105 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:39:45.063048    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:39:45.157643 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:39:45.157608    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:41:12.732227 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:41:12.732191    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:41:23.301423 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:41:23.301386    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:41:59.195865 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:41:59.195824    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:42:00.707783 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:42:00.707744    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:43:30.270377 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:43:30.270347    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:43:30.279929 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:43:30.279907    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:45:00.138114 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:00.138078    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742885-sdgbf"]
Jul 20 18:45:00.141786 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:00.141770    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742885-sdgbf"
Jul 20 18:45:00.144515 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:00.144487    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w8qkh\""
Jul 20 18:45:00.151630 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:00.151613    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742885-sdgbf"
Jul 20 18:45:00.162146 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:00.162117    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742885-sdgbf"]
Jul 20 18:45:00.295164 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:00.295138    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742885-sdgbf"]
Jul 20 18:45:00.297927 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:45:00.297896    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb176261d_8eea_4b2c_93b6_1aeb438cdd99.slice/crio-9e8a131515a7eceddd2751559484f17a07280f683e1a7fdf502f3e8bf043e1e5 WatchSource:0}: Error finding container 9e8a131515a7eceddd2751559484f17a07280f683e1a7fdf502f3e8bf043e1e5: Status 404 returned error can't find the container with id 9e8a131515a7eceddd2751559484f17a07280f683e1a7fdf502f3e8bf043e1e5
Jul 20 18:45:00.513521 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:00.513484    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742885-sdgbf" event={"ID":"b176261d-8eea-4b2c-93b6-1aeb438cdd99","Type":"ContainerStarted","Data":"9e8a131515a7eceddd2751559484f17a07280f683e1a7fdf502f3e8bf043e1e5"}
Jul 20 18:45:01.520153 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:01.520121    2569 generic.go:356] "Generic (PLEG): container finished" podID="b176261d-8eea-4b2c-93b6-1aeb438cdd99" containerID="209374f0d64656a31978c7ed69618a788b97d6ede5494c30c1fbdcf8d94deb5c" exitCode=0
Jul 20 18:45:01.520542 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:01.520214    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742885-sdgbf" event={"ID":"b176261d-8eea-4b2c-93b6-1aeb438cdd99","Type":"ContainerDied","Data":"209374f0d64656a31978c7ed69618a788b97d6ede5494c30c1fbdcf8d94deb5c"}
Jul 20 18:45:02.659246 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:02.659217    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742885-sdgbf"
Jul 20 18:45:03.531388 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:03.531359    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742885-sdgbf"
Jul 20 18:45:03.531388 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:03.531387    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742885-sdgbf" event={"ID":"b176261d-8eea-4b2c-93b6-1aeb438cdd99","Type":"ContainerDied","Data":"9e8a131515a7eceddd2751559484f17a07280f683e1a7fdf502f3e8bf043e1e5"}
Jul 20 18:45:03.531608 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:03.531438    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e8a131515a7eceddd2751559484f17a07280f683e1a7fdf502f3e8bf043e1e5"
Jul 20 18:45:16.337245 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:16.337170    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:30.165405 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:30.165340    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:38.393873 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:38.393836    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:43.473667 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:43.473633    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:43.742980 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:43.742910    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:47.397712 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:47.397645    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:47.511439 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:47.511386    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:53.293391 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:53.293358    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:53.390821 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:53.390777    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:53.512772 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:53.512720    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:53.874306 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:53.874261    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:53.998936 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:53.998908    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:55.841109 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:55.841075    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:55.924751 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:55.924716    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:55.928733 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:55.928701    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"]
Jul 20 18:45:55.929195 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:55.929177    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b176261d-8eea-4b2c-93b6-1aeb438cdd99" containerName="cleanup"
Jul 20 18:45:55.929296 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:55.929198    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b176261d-8eea-4b2c-93b6-1aeb438cdd99" containerName="cleanup"
Jul 20 18:45:55.929363 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:55.929317    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b176261d-8eea-4b2c-93b6-1aeb438cdd99" containerName="cleanup"
Jul 20 18:45:55.932504 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:55.932486    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"
Jul 20 18:45:55.934841 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:55.934823    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5b7f2845-serving-cert\""
Jul 20 18:45:55.934940 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:55.934875    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vvgb2\""
Jul 20 18:45:55.944217 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:55.944193    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"]
Jul 20 18:45:55.970615 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:55.970583    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/347dc84c-1f45-4881-9900-a0a7536830c9-openshift-service-ca\") pod \"maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb\" (UID: \"347dc84c-1f45-4881-9900-a0a7536830c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"
Jul 20 18:45:55.970743 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:55.970629    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/347dc84c-1f45-4881-9900-a0a7536830c9-maas-api-tls\") pod \"maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb\" (UID: \"347dc84c-1f45-4881-9900-a0a7536830c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"
Jul 20 18:45:55.970743 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:55.970698    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkr7k\" (UniqueName: \"kubernetes.io/projected/347dc84c-1f45-4881-9900-a0a7536830c9-kube-api-access-pkr7k\") pod \"maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb\" (UID: \"347dc84c-1f45-4881-9900-a0a7536830c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"
Jul 20 18:45:56.004288 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:56.004256    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:56.071356 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:56.071320    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkr7k\" (UniqueName: \"kubernetes.io/projected/347dc84c-1f45-4881-9900-a0a7536830c9-kube-api-access-pkr7k\") pod \"maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb\" (UID: \"347dc84c-1f45-4881-9900-a0a7536830c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"
Jul 20 18:45:56.071537 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:56.071422    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/347dc84c-1f45-4881-9900-a0a7536830c9-openshift-service-ca\") pod \"maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb\" (UID: \"347dc84c-1f45-4881-9900-a0a7536830c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"
Jul 20 18:45:56.071537 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:56.071456    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/347dc84c-1f45-4881-9900-a0a7536830c9-maas-api-tls\") pod \"maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb\" (UID: \"347dc84c-1f45-4881-9900-a0a7536830c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"
Jul 20 18:45:56.071622 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:45:56.071571    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-serving-cert: secret "maas-api-e2e-ait-5b7f2845-serving-cert" not found
Jul 20 18:45:56.071661 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:45:56.071639    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/347dc84c-1f45-4881-9900-a0a7536830c9-maas-api-tls podName:347dc84c-1f45-4881-9900-a0a7536830c9 nodeName:}" failed. No retries permitted until 2026-07-20 18:45:56.571620322 +0000 UTC m=+1953.405592693 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/347dc84c-1f45-4881-9900-a0a7536830c9-maas-api-tls") pod "maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb" (UID: "347dc84c-1f45-4881-9900-a0a7536830c9") : secret "maas-api-e2e-ait-5b7f2845-serving-cert" not found
Jul 20 18:45:56.072149 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:56.072130    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/347dc84c-1f45-4881-9900-a0a7536830c9-openshift-service-ca\") pod \"maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb\" (UID: \"347dc84c-1f45-4881-9900-a0a7536830c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"
Jul 20 18:45:56.080710 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:56.080684    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkr7k\" (UniqueName: \"kubernetes.io/projected/347dc84c-1f45-4881-9900-a0a7536830c9-kube-api-access-pkr7k\") pod \"maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb\" (UID: \"347dc84c-1f45-4881-9900-a0a7536830c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"
Jul 20 18:45:56.503075 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:56.503036    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:56.577758 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:56.577717    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/347dc84c-1f45-4881-9900-a0a7536830c9-maas-api-tls\") pod \"maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb\" (UID: \"347dc84c-1f45-4881-9900-a0a7536830c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"
Jul 20 18:45:56.580165 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:56.580144    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/347dc84c-1f45-4881-9900-a0a7536830c9-maas-api-tls\") pod \"maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb\" (UID: \"347dc84c-1f45-4881-9900-a0a7536830c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"
Jul 20 18:45:56.760482 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:56.760400    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:56.844840 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:56.844807    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"
Jul 20 18:45:56.859199 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:56.859173    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:45:57.187234 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:57.187211    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"]
Jul 20 18:45:57.189331 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:45:57.189303    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod347dc84c_1f45_4881_9900_a0a7536830c9.slice/crio-65927445c8d649c41ef033dc030bdf9894c1f7a637ad67c90b015b14b9139c78 WatchSource:0}: Error finding container 65927445c8d649c41ef033dc030bdf9894c1f7a637ad67c90b015b14b9139c78: Status 404 returned error can't find the container with id 65927445c8d649c41ef033dc030bdf9894c1f7a637ad67c90b015b14b9139c78
Jul 20 18:45:57.190781 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:57.190759    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 18:45:57.791670 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:57.791634    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb" event={"ID":"347dc84c-1f45-4881-9900-a0a7536830c9","Type":"ContainerStarted","Data":"81fc4b1cea50c686dc0a5add5ff1f75c4d0b1319e98c1ab53242194b8efabbef"}
Jul 20 18:45:57.791846 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:57.791696    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb" event={"ID":"347dc84c-1f45-4881-9900-a0a7536830c9","Type":"ContainerStarted","Data":"65927445c8d649c41ef033dc030bdf9894c1f7a637ad67c90b015b14b9139c78"}
Jul 20 18:45:57.791846 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:57.791819    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"
Jul 20 18:45:57.809795 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:45:57.809740    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb" podStartSLOduration=2.305718243 podStartE2EDuration="2.809724733s" totalImagesPullingTime="504.00649ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:45:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:45:57.807559154 +0000 UTC m=+1954.641531559" watchObservedRunningTime="2026-07-20 18:45:57.809724733 +0000 UTC m=+1954.643697120"
Jul 20 18:46:03.801179 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:03.801152    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"
Jul 20 18:46:21.367230 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:21.367137    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp"]
Jul 20 18:46:21.371123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:21.371101    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp"
Jul 20 18:46:21.373743 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:21.373721    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w8qkh\""
Jul 20 18:46:21.379507 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:21.379462    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp"]
Jul 20 18:46:21.513485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:21.513444    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4cb7a66-9dd9-43f4-89bb-196d2056a05c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp\" (UID: \"e4cb7a66-9dd9-43f4-89bb-196d2056a05c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp"
Jul 20 18:46:21.614398 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:21.614362    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4cb7a66-9dd9-43f4-89bb-196d2056a05c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp\" (UID: \"e4cb7a66-9dd9-43f4-89bb-196d2056a05c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp"
Jul 20 18:46:21.614998 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:21.614979    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4cb7a66-9dd9-43f4-89bb-196d2056a05c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp\" (UID: \"e4cb7a66-9dd9-43f4-89bb-196d2056a05c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp"
Jul 20 18:46:21.683460 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:21.683359    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp"
Jul 20 18:46:21.819591 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:21.819566    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp"]
Jul 20 18:46:21.820811 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:46:21.820788    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4cb7a66_9dd9_43f4_89bb_196d2056a05c.slice/crio-9456e4f1eba3eb976f1b0cba5dfbe710dab3fd5aaf36a30f35e35fdcf1ca7c19 WatchSource:0}: Error finding container 9456e4f1eba3eb976f1b0cba5dfbe710dab3fd5aaf36a30f35e35fdcf1ca7c19: Status 404 returned error can't find the container with id 9456e4f1eba3eb976f1b0cba5dfbe710dab3fd5aaf36a30f35e35fdcf1ca7c19
Jul 20 18:46:21.915358 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:21.915322    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp" event={"ID":"e4cb7a66-9dd9-43f4-89bb-196d2056a05c","Type":"ContainerStarted","Data":"9456e4f1eba3eb976f1b0cba5dfbe710dab3fd5aaf36a30f35e35fdcf1ca7c19"}
Jul 20 18:46:22.922234 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:22.922142    2569 generic.go:356] "Generic (PLEG): container finished" podID="e4cb7a66-9dd9-43f4-89bb-196d2056a05c" containerID="f1e676fddd2c7d6154227c19c2c44d0df86eaba90203abe437519ae9fca315a9" exitCode=0
Jul 20 18:46:22.922638 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:22.922228    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp" event={"ID":"e4cb7a66-9dd9-43f4-89bb-196d2056a05c","Type":"ContainerDied","Data":"f1e676fddd2c7d6154227c19c2c44d0df86eaba90203abe437519ae9fca315a9"}
Jul 20 18:46:24.056291 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:24.056268    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp"
Jul 20 18:46:24.134447 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:24.134410    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4cb7a66-9dd9-43f4-89bb-196d2056a05c-maas-api-service-ca\") pod \"e4cb7a66-9dd9-43f4-89bb-196d2056a05c\" (UID: \"e4cb7a66-9dd9-43f4-89bb-196d2056a05c\") "
Jul 20 18:46:24.134785 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:24.134761    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4cb7a66-9dd9-43f4-89bb-196d2056a05c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e4cb7a66-9dd9-43f4-89bb-196d2056a05c" (UID: "e4cb7a66-9dd9-43f4-89bb-196d2056a05c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:46:24.235489 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:24.235414    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4cb7a66-9dd9-43f4-89bb-196d2056a05c-maas-api-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:46:24.933812 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:24.933778    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp" event={"ID":"e4cb7a66-9dd9-43f4-89bb-196d2056a05c","Type":"ContainerDied","Data":"9456e4f1eba3eb976f1b0cba5dfbe710dab3fd5aaf36a30f35e35fdcf1ca7c19"}
Jul 20 18:46:24.933812 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:24.933811    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp"
Jul 20 18:46:24.934050 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:24.933826    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9456e4f1eba3eb976f1b0cba5dfbe710dab3fd5aaf36a30f35e35fdcf1ca7c19"
Jul 20 18:46:31.870415 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:31.870380    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"]
Jul 20 18:46:31.870958 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:31.870585    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb" podUID="347dc84c-1f45-4881-9900-a0a7536830c9" containerName="maas-api" containerID="cri-o://81fc4b1cea50c686dc0a5add5ff1f75c4d0b1319e98c1ab53242194b8efabbef" gracePeriod=30
Jul 20 18:46:31.898722 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:31.898687    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:46:32.120733 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.120674    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"
Jul 20 18:46:32.208453 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.208417    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkr7k\" (UniqueName: \"kubernetes.io/projected/347dc84c-1f45-4881-9900-a0a7536830c9-kube-api-access-pkr7k\") pod \"347dc84c-1f45-4881-9900-a0a7536830c9\" (UID: \"347dc84c-1f45-4881-9900-a0a7536830c9\") "
Jul 20 18:46:32.208647 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.208520    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/347dc84c-1f45-4881-9900-a0a7536830c9-maas-api-tls\") pod \"347dc84c-1f45-4881-9900-a0a7536830c9\" (UID: \"347dc84c-1f45-4881-9900-a0a7536830c9\") "
Jul 20 18:46:32.208647 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.208554    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/347dc84c-1f45-4881-9900-a0a7536830c9-openshift-service-ca\") pod \"347dc84c-1f45-4881-9900-a0a7536830c9\" (UID: \"347dc84c-1f45-4881-9900-a0a7536830c9\") "
Jul 20 18:46:32.208964 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.208939    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/347dc84c-1f45-4881-9900-a0a7536830c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "347dc84c-1f45-4881-9900-a0a7536830c9" (UID: "347dc84c-1f45-4881-9900-a0a7536830c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:46:32.210610 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.210581    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/347dc84c-1f45-4881-9900-a0a7536830c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "347dc84c-1f45-4881-9900-a0a7536830c9" (UID: "347dc84c-1f45-4881-9900-a0a7536830c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:46:32.210720 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.210627    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/347dc84c-1f45-4881-9900-a0a7536830c9-kube-api-access-pkr7k" (OuterVolumeSpecName: "kube-api-access-pkr7k") pod "347dc84c-1f45-4881-9900-a0a7536830c9" (UID: "347dc84c-1f45-4881-9900-a0a7536830c9"). InnerVolumeSpecName "kube-api-access-pkr7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:46:32.309291 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.309246    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/347dc84c-1f45-4881-9900-a0a7536830c9-openshift-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:46:32.309291 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.309288    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkr7k\" (UniqueName: \"kubernetes.io/projected/347dc84c-1f45-4881-9900-a0a7536830c9-kube-api-access-pkr7k\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:46:32.309291 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.309298    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/347dc84c-1f45-4881-9900-a0a7536830c9-maas-api-tls\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:46:32.973694 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.973655    2569 generic.go:356] "Generic (PLEG): container finished" podID="347dc84c-1f45-4881-9900-a0a7536830c9" containerID="81fc4b1cea50c686dc0a5add5ff1f75c4d0b1319e98c1ab53242194b8efabbef" exitCode=0
Jul 20 18:46:32.974129 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.973704    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"
Jul 20 18:46:32.974129 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.973733    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb" event={"ID":"347dc84c-1f45-4881-9900-a0a7536830c9","Type":"ContainerDied","Data":"81fc4b1cea50c686dc0a5add5ff1f75c4d0b1319e98c1ab53242194b8efabbef"}
Jul 20 18:46:32.974129 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.973786    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb" event={"ID":"347dc84c-1f45-4881-9900-a0a7536830c9","Type":"ContainerDied","Data":"65927445c8d649c41ef033dc030bdf9894c1f7a637ad67c90b015b14b9139c78"}
Jul 20 18:46:32.974129 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.973808    2569 scope.go:117] "RemoveContainer" containerID="81fc4b1cea50c686dc0a5add5ff1f75c4d0b1319e98c1ab53242194b8efabbef"
Jul 20 18:46:32.983070 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.983031    2569 scope.go:117] "RemoveContainer" containerID="81fc4b1cea50c686dc0a5add5ff1f75c4d0b1319e98c1ab53242194b8efabbef"
Jul 20 18:46:32.983338 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:46:32.983319    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81fc4b1cea50c686dc0a5add5ff1f75c4d0b1319e98c1ab53242194b8efabbef\": container with ID starting with 81fc4b1cea50c686dc0a5add5ff1f75c4d0b1319e98c1ab53242194b8efabbef not found: ID does not exist" containerID="81fc4b1cea50c686dc0a5add5ff1f75c4d0b1319e98c1ab53242194b8efabbef"
Jul 20 18:46:32.983392 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.983347    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81fc4b1cea50c686dc0a5add5ff1f75c4d0b1319e98c1ab53242194b8efabbef"} err="failed to get container status \"81fc4b1cea50c686dc0a5add5ff1f75c4d0b1319e98c1ab53242194b8efabbef\": rpc error: code = NotFound desc = could not find container \"81fc4b1cea50c686dc0a5add5ff1f75c4d0b1319e98c1ab53242194b8efabbef\": container with ID starting with 81fc4b1cea50c686dc0a5add5ff1f75c4d0b1319e98c1ab53242194b8efabbef not found: ID does not exist"
Jul 20 18:46:32.998873 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:32.998846    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"]
Jul 20 18:46:33.004024 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:33.004000    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5b7f2845-84d8c7b88-sv4hb"]
Jul 20 18:46:33.760195 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:33.760164    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="347dc84c-1f45-4881-9900-a0a7536830c9" path="/var/lib/kubelet/pods/347dc84c-1f45-4881-9900-a0a7536830c9/volumes"
Jul 20 18:46:33.876096 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:33.876017    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:46:36.452626 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:36.452580    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp"]
Jul 20 18:46:36.455503 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:36.455476    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5b7f2845-qrfhp"]
Jul 20 18:46:36.456387 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:36.456366    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:46:36.610768 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:36.610732    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:46:36.859721 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:36.859683    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:46:37.177046 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:37.176959    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:46:37.382594 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:37.382559    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:46:37.469425 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:37.469338    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:46:37.659018 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:37.658987    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:46:37.759014 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:37.758974    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4cb7a66-9dd9-43f4-89bb-196d2056a05c" path="/var/lib/kubelet/pods/e4cb7a66-9dd9-43f4-89bb-196d2056a05c/volumes"
Jul 20 18:46:38.016945 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:38.016866    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:46:39.663412 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.663378    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:46:39.744859 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.744829    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"]
Jul 20 18:46:39.745300 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.745285    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4cb7a66-9dd9-43f4-89bb-196d2056a05c" containerName="revoke-keys"
Jul 20 18:46:39.745350 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.745302    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4cb7a66-9dd9-43f4-89bb-196d2056a05c" containerName="revoke-keys"
Jul 20 18:46:39.745350 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.745339    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="347dc84c-1f45-4881-9900-a0a7536830c9" containerName="maas-api"
Jul 20 18:46:39.745350 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.745346    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="347dc84c-1f45-4881-9900-a0a7536830c9" containerName="maas-api"
Jul 20 18:46:39.745501 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.745416    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4cb7a66-9dd9-43f4-89bb-196d2056a05c" containerName="revoke-keys"
Jul 20 18:46:39.745501 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.745429    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="347dc84c-1f45-4881-9900-a0a7536830c9" containerName="maas-api"
Jul 20 18:46:39.749814 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.749793    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"
Jul 20 18:46:39.752459 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.752426    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-4cc2704b-serving-cert\""
Jul 20 18:46:39.752587 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.752471    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vvgb2\""
Jul 20 18:46:39.762144 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.762121    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"]
Jul 20 18:46:39.784617 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.784584    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:46:39.875016 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.874988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg7zt\" (UniqueName: \"kubernetes.io/projected/fbb7f434-d6e0-445f-8f93-a0461f24274f-kube-api-access-pg7zt\") pod \"maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7\" (UID: \"fbb7f434-d6e0-445f-8f93-a0461f24274f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"
Jul 20 18:46:39.875202 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.875154    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbb7f434-d6e0-445f-8f93-a0461f24274f-openshift-service-ca\") pod \"maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7\" (UID: \"fbb7f434-d6e0-445f-8f93-a0461f24274f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"
Jul 20 18:46:39.875281 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.875197    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbb7f434-d6e0-445f-8f93-a0461f24274f-maas-api-tls\") pod \"maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7\" (UID: \"fbb7f434-d6e0-445f-8f93-a0461f24274f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"
Jul 20 18:46:39.976310 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.976232    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbb7f434-d6e0-445f-8f93-a0461f24274f-openshift-service-ca\") pod \"maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7\" (UID: \"fbb7f434-d6e0-445f-8f93-a0461f24274f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"
Jul 20 18:46:39.976310 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.976266    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbb7f434-d6e0-445f-8f93-a0461f24274f-maas-api-tls\") pod \"maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7\" (UID: \"fbb7f434-d6e0-445f-8f93-a0461f24274f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"
Jul 20 18:46:39.976492 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.976324    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg7zt\" (UniqueName: \"kubernetes.io/projected/fbb7f434-d6e0-445f-8f93-a0461f24274f-kube-api-access-pg7zt\") pod \"maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7\" (UID: \"fbb7f434-d6e0-445f-8f93-a0461f24274f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"
Jul 20 18:46:39.976492 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:46:39.976445    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-serving-cert: secret "maas-api-e2e-migrate-4cc2704b-serving-cert" not found
Jul 20 18:46:39.976577 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:46:39.976522    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb7f434-d6e0-445f-8f93-a0461f24274f-maas-api-tls podName:fbb7f434-d6e0-445f-8f93-a0461f24274f nodeName:}" failed. No retries permitted until 2026-07-20 18:46:40.476502753 +0000 UTC m=+1997.310475121 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fbb7f434-d6e0-445f-8f93-a0461f24274f-maas-api-tls") pod "maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7" (UID: "fbb7f434-d6e0-445f-8f93-a0461f24274f") : secret "maas-api-e2e-migrate-4cc2704b-serving-cert" not found
Jul 20 18:46:39.977012 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.976987    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbb7f434-d6e0-445f-8f93-a0461f24274f-openshift-service-ca\") pod \"maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7\" (UID: \"fbb7f434-d6e0-445f-8f93-a0461f24274f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"
Jul 20 18:46:39.987420 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:39.987392    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg7zt\" (UniqueName: \"kubernetes.io/projected/fbb7f434-d6e0-445f-8f93-a0461f24274f-kube-api-access-pg7zt\") pod \"maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7\" (UID: \"fbb7f434-d6e0-445f-8f93-a0461f24274f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"
Jul 20 18:46:40.296040 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:40.296006    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:46:40.396084 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:40.396037    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:46:40.482462 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:40.482425    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbb7f434-d6e0-445f-8f93-a0461f24274f-maas-api-tls\") pod \"maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7\" (UID: \"fbb7f434-d6e0-445f-8f93-a0461f24274f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"
Jul 20 18:46:40.484837 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:40.484806    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbb7f434-d6e0-445f-8f93-a0461f24274f-maas-api-tls\") pod \"maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7\" (UID: \"fbb7f434-d6e0-445f-8f93-a0461f24274f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"
Jul 20 18:46:40.509010 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:40.508967    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:46:40.662722 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:40.662628    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"
Jul 20 18:46:40.809481 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:40.809455    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"]
Jul 20 18:46:40.811717 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:46:40.811690    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbb7f434_d6e0_445f_8f93_a0461f24274f.slice/crio-bbde40801927c82ae5c3e780c9f900224b63bcffef2a713c79cb414326abc914 WatchSource:0}: Error finding container bbde40801927c82ae5c3e780c9f900224b63bcffef2a713c79cb414326abc914: Status 404 returned error can't find the container with id bbde40801927c82ae5c3e780c9f900224b63bcffef2a713c79cb414326abc914
Jul 20 18:46:41.014979 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:41.014938    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7" event={"ID":"fbb7f434-d6e0-445f-8f93-a0461f24274f","Type":"ContainerStarted","Data":"bbde40801927c82ae5c3e780c9f900224b63bcffef2a713c79cb414326abc914"}
Jul 20 18:46:42.021570 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:42.021534    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7" event={"ID":"fbb7f434-d6e0-445f-8f93-a0461f24274f","Type":"ContainerStarted","Data":"a2cf858e94bd7fd6dc50f8c00ac5cd53babf62ca4603a4c0f3d7943079a85972"}
Jul 20 18:46:42.022044 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:42.021743    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"
Jul 20 18:46:42.042010 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:42.041961    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7" podStartSLOduration=2.676949059 podStartE2EDuration="3.041944915s" totalImagesPullingTime="364.995856ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:46:42.040806972 +0000 UTC m=+1998.874779361" watchObservedRunningTime="2026-07-20 18:46:42.041944915 +0000 UTC m=+1998.875917302"
Jul 20 18:46:48.032854 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:46:48.032820    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"
Jul 20 18:47:04.678509 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:04.678467    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57"]
Jul 20 18:47:04.682338 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:04.682311    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57"
Jul 20 18:47:04.684825 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:04.684805    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w8qkh\""
Jul 20 18:47:04.689670 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:04.689647    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57"]
Jul 20 18:47:04.715437 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:04.715408    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42ea55e9-5cef-4776-bbae-9c3d9582ac86-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57\" (UID: \"42ea55e9-5cef-4776-bbae-9c3d9582ac86\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57"
Jul 20 18:47:04.816338 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:04.816305    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42ea55e9-5cef-4776-bbae-9c3d9582ac86-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57\" (UID: \"42ea55e9-5cef-4776-bbae-9c3d9582ac86\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57"
Jul 20 18:47:04.816982 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:04.816952    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42ea55e9-5cef-4776-bbae-9c3d9582ac86-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57\" (UID: \"42ea55e9-5cef-4776-bbae-9c3d9582ac86\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57"
Jul 20 18:47:04.996224 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:04.996183    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57"
Jul 20 18:47:05.131924 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:05.131896    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57"]
Jul 20 18:47:05.133961 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:47:05.133923    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42ea55e9_5cef_4776_bbae_9c3d9582ac86.slice/crio-7a2d9ea3d78c42b5e71ea27463a5d8452ed67f358453475745f80cde715aa42e WatchSource:0}: Error finding container 7a2d9ea3d78c42b5e71ea27463a5d8452ed67f358453475745f80cde715aa42e: Status 404 returned error can't find the container with id 7a2d9ea3d78c42b5e71ea27463a5d8452ed67f358453475745f80cde715aa42e
Jul 20 18:47:05.142432 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:05.142402    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57" event={"ID":"42ea55e9-5cef-4776-bbae-9c3d9582ac86","Type":"ContainerStarted","Data":"7a2d9ea3d78c42b5e71ea27463a5d8452ed67f358453475745f80cde715aa42e"}
Jul 20 18:47:06.148690 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:06.148599    2569 generic.go:356] "Generic (PLEG): container finished" podID="42ea55e9-5cef-4776-bbae-9c3d9582ac86" containerID="0b92c342b02ce646e4fcdd55ecdb6972ac7897627ead48671e5c9da4eb52db87" exitCode=0
Jul 20 18:47:06.148690 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:06.148670    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57" event={"ID":"42ea55e9-5cef-4776-bbae-9c3d9582ac86","Type":"ContainerDied","Data":"0b92c342b02ce646e4fcdd55ecdb6972ac7897627ead48671e5c9da4eb52db87"}
Jul 20 18:47:07.293108 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:07.293082    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57"
Jul 20 18:47:07.338759 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:07.338716    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42ea55e9-5cef-4776-bbae-9c3d9582ac86-maas-api-service-ca\") pod \"42ea55e9-5cef-4776-bbae-9c3d9582ac86\" (UID: \"42ea55e9-5cef-4776-bbae-9c3d9582ac86\") "
Jul 20 18:47:07.339082 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:07.339044    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42ea55e9-5cef-4776-bbae-9c3d9582ac86-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "42ea55e9-5cef-4776-bbae-9c3d9582ac86" (UID: "42ea55e9-5cef-4776-bbae-9c3d9582ac86"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:47:07.439524 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:07.439441    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42ea55e9-5cef-4776-bbae-9c3d9582ac86-maas-api-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:47:08.159460 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:08.159377    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57"
Jul 20 18:47:08.159460 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:08.159420    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57" event={"ID":"42ea55e9-5cef-4776-bbae-9c3d9582ac86","Type":"ContainerDied","Data":"7a2d9ea3d78c42b5e71ea27463a5d8452ed67f358453475745f80cde715aa42e"}
Jul 20 18:47:08.159645 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:08.159470    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a2d9ea3d78c42b5e71ea27463a5d8452ed67f358453475745f80cde715aa42e"
Jul 20 18:47:09.044948 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:09.044915    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:15.298418 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:15.298354    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"]
Jul 20 18:47:15.298984 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:15.298594    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7" podUID="fbb7f434-d6e0-445f-8f93-a0461f24274f" containerName="maas-api" containerID="cri-o://a2cf858e94bd7fd6dc50f8c00ac5cd53babf62ca4603a4c0f3d7943079a85972" gracePeriod=30
Jul 20 18:47:15.334179 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:15.334135    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:15.558108 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:15.558028    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"
Jul 20 18:47:15.615608 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:15.615575    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbb7f434-d6e0-445f-8f93-a0461f24274f-maas-api-tls\") pod \"fbb7f434-d6e0-445f-8f93-a0461f24274f\" (UID: \"fbb7f434-d6e0-445f-8f93-a0461f24274f\") "
Jul 20 18:47:15.615768 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:15.615626    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pg7zt\" (UniqueName: \"kubernetes.io/projected/fbb7f434-d6e0-445f-8f93-a0461f24274f-kube-api-access-pg7zt\") pod \"fbb7f434-d6e0-445f-8f93-a0461f24274f\" (UID: \"fbb7f434-d6e0-445f-8f93-a0461f24274f\") "
Jul 20 18:47:15.615768 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:15.615698    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbb7f434-d6e0-445f-8f93-a0461f24274f-openshift-service-ca\") pod \"fbb7f434-d6e0-445f-8f93-a0461f24274f\" (UID: \"fbb7f434-d6e0-445f-8f93-a0461f24274f\") "
Jul 20 18:47:15.616127 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:15.616094    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbb7f434-d6e0-445f-8f93-a0461f24274f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fbb7f434-d6e0-445f-8f93-a0461f24274f" (UID: "fbb7f434-d6e0-445f-8f93-a0461f24274f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:47:15.617639 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:15.617618    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbb7f434-d6e0-445f-8f93-a0461f24274f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fbb7f434-d6e0-445f-8f93-a0461f24274f" (UID: "fbb7f434-d6e0-445f-8f93-a0461f24274f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:47:15.617742 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:15.617726    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbb7f434-d6e0-445f-8f93-a0461f24274f-kube-api-access-pg7zt" (OuterVolumeSpecName: "kube-api-access-pg7zt") pod "fbb7f434-d6e0-445f-8f93-a0461f24274f" (UID: "fbb7f434-d6e0-445f-8f93-a0461f24274f"). InnerVolumeSpecName "kube-api-access-pg7zt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:47:15.716607 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:15.716577    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbb7f434-d6e0-445f-8f93-a0461f24274f-openshift-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:47:15.716607 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:15.716603    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbb7f434-d6e0-445f-8f93-a0461f24274f-maas-api-tls\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:47:15.716607 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:15.716612    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pg7zt\" (UniqueName: \"kubernetes.io/projected/fbb7f434-d6e0-445f-8f93-a0461f24274f-kube-api-access-pg7zt\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:47:16.207526 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:16.207494    2569 generic.go:356] "Generic (PLEG): container finished" podID="fbb7f434-d6e0-445f-8f93-a0461f24274f" containerID="a2cf858e94bd7fd6dc50f8c00ac5cd53babf62ca4603a4c0f3d7943079a85972" exitCode=0
Jul 20 18:47:16.207705 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:16.207557    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7" event={"ID":"fbb7f434-d6e0-445f-8f93-a0461f24274f","Type":"ContainerDied","Data":"a2cf858e94bd7fd6dc50f8c00ac5cd53babf62ca4603a4c0f3d7943079a85972"}
Jul 20 18:47:16.207705 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:16.207618    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7" event={"ID":"fbb7f434-d6e0-445f-8f93-a0461f24274f","Type":"ContainerDied","Data":"bbde40801927c82ae5c3e780c9f900224b63bcffef2a713c79cb414326abc914"}
Jul 20 18:47:16.207705 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:16.207559    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"
Jul 20 18:47:16.207705 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:16.207639    2569 scope.go:117] "RemoveContainer" containerID="a2cf858e94bd7fd6dc50f8c00ac5cd53babf62ca4603a4c0f3d7943079a85972"
Jul 20 18:47:16.219765 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:16.219744    2569 scope.go:117] "RemoveContainer" containerID="a2cf858e94bd7fd6dc50f8c00ac5cd53babf62ca4603a4c0f3d7943079a85972"
Jul 20 18:47:16.220039 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:47:16.220020    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2cf858e94bd7fd6dc50f8c00ac5cd53babf62ca4603a4c0f3d7943079a85972\": container with ID starting with a2cf858e94bd7fd6dc50f8c00ac5cd53babf62ca4603a4c0f3d7943079a85972 not found: ID does not exist" containerID="a2cf858e94bd7fd6dc50f8c00ac5cd53babf62ca4603a4c0f3d7943079a85972"
Jul 20 18:47:16.220125 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:16.220045    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2cf858e94bd7fd6dc50f8c00ac5cd53babf62ca4603a4c0f3d7943079a85972"} err="failed to get container status \"a2cf858e94bd7fd6dc50f8c00ac5cd53babf62ca4603a4c0f3d7943079a85972\": rpc error: code = NotFound desc = could not find container \"a2cf858e94bd7fd6dc50f8c00ac5cd53babf62ca4603a4c0f3d7943079a85972\": container with ID starting with a2cf858e94bd7fd6dc50f8c00ac5cd53babf62ca4603a4c0f3d7943079a85972 not found: ID does not exist"
Jul 20 18:47:16.225785 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:16.225763    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"]
Jul 20 18:47:16.230323 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:16.230300    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4cc2704b-c6fff8f7f-bgzt7"]
Jul 20 18:47:17.759724 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:17.759689    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbb7f434-d6e0-445f-8f93-a0461f24274f" path="/var/lib/kubelet/pods/fbb7f434-d6e0-445f-8f93-a0461f24274f/volumes"
Jul 20 18:47:19.817073 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:19.817008    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:19.817590 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:19.817569    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57"]
Jul 20 18:47:19.819429 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:19.819392    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4cc2704b-5dr57"]
Jul 20 18:47:19.980555 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:19.980524    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:20.226091 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:20.225991    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:20.448190 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:20.448154    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:20.785895 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:20.785863    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:21.150411 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:21.150330    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:21.758788 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:21.758755    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42ea55e9-5cef-4776-bbae-9c3d9582ac86" path="/var/lib/kubelet/pods/42ea55e9-5cef-4776-bbae-9c3d9582ac86/volumes"
Jul 20 18:47:23.428207 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.428174    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:23.513359 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.513324    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"]
Jul 20 18:47:23.513927 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.513909    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42ea55e9-5cef-4776-bbae-9c3d9582ac86" containerName="revoke-keys"
Jul 20 18:47:23.514016 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.513930    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="42ea55e9-5cef-4776-bbae-9c3d9582ac86" containerName="revoke-keys"
Jul 20 18:47:23.514016 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.513967    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbb7f434-d6e0-445f-8f93-a0461f24274f" containerName="maas-api"
Jul 20 18:47:23.514016 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.513977    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbb7f434-d6e0-445f-8f93-a0461f24274f" containerName="maas-api"
Jul 20 18:47:23.514140 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.514097    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="42ea55e9-5cef-4776-bbae-9c3d9582ac86" containerName="revoke-keys"
Jul 20 18:47:23.514140 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.514110    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbb7f434-d6e0-445f-8f93-a0461f24274f" containerName="maas-api"
Jul 20 18:47:23.515618 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.515594    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:23.518751 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.518735    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"
Jul 20 18:47:23.521786 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.521767    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vvgb2\""
Jul 20 18:47:23.522144 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.522126    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-bae9791c-serving-cert\""
Jul 20 18:47:23.527722 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.527698    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"]
Jul 20 18:47:23.588109 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.588075    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rplc\" (UniqueName: \"kubernetes.io/projected/5b428e60-10ec-430f-9ed0-9cec7a4ef272-kube-api-access-6rplc\") pod \"maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds\" (UID: \"5b428e60-10ec-430f-9ed0-9cec7a4ef272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"
Jul 20 18:47:23.588273 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.588209    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b428e60-10ec-430f-9ed0-9cec7a4ef272-maas-api-tls\") pod \"maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds\" (UID: \"5b428e60-10ec-430f-9ed0-9cec7a4ef272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"
Jul 20 18:47:23.588273 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.588266    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b428e60-10ec-430f-9ed0-9cec7a4ef272-openshift-service-ca\") pod \"maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds\" (UID: \"5b428e60-10ec-430f-9ed0-9cec7a4ef272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"
Jul 20 18:47:23.594500 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.594475    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:23.689187 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.689099    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b428e60-10ec-430f-9ed0-9cec7a4ef272-maas-api-tls\") pod \"maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds\" (UID: \"5b428e60-10ec-430f-9ed0-9cec7a4ef272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"
Jul 20 18:47:23.689187 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.689153    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b428e60-10ec-430f-9ed0-9cec7a4ef272-openshift-service-ca\") pod \"maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds\" (UID: \"5b428e60-10ec-430f-9ed0-9cec7a4ef272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"
Jul 20 18:47:23.689382 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.689288    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rplc\" (UniqueName: \"kubernetes.io/projected/5b428e60-10ec-430f-9ed0-9cec7a4ef272-kube-api-access-6rplc\") pod \"maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds\" (UID: \"5b428e60-10ec-430f-9ed0-9cec7a4ef272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"
Jul 20 18:47:23.689723 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.689706    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b428e60-10ec-430f-9ed0-9cec7a4ef272-openshift-service-ca\") pod \"maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds\" (UID: \"5b428e60-10ec-430f-9ed0-9cec7a4ef272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"
Jul 20 18:47:23.693465 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.693449    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-bae9791c-serving-cert\""
Jul 20 18:47:23.702105 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.702083    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b428e60-10ec-430f-9ed0-9cec7a4ef272-maas-api-tls\") pod \"maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds\" (UID: \"5b428e60-10ec-430f-9ed0-9cec7a4ef272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"
Jul 20 18:47:23.702623 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.702601    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rplc\" (UniqueName: \"kubernetes.io/projected/5b428e60-10ec-430f-9ed0-9cec7a4ef272-kube-api-access-6rplc\") pod \"maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds\" (UID: \"5b428e60-10ec-430f-9ed0-9cec7a4ef272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"
Jul 20 18:47:23.719257 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.719223    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:23.835213 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.835190    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vvgb2\""
Jul 20 18:47:23.840309 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.840280    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"
Jul 20 18:47:23.986608 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:23.986580    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"]
Jul 20 18:47:23.988290 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:47:23.988264    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b428e60_10ec_430f_9ed0_9cec7a4ef272.slice/crio-6f360fdf7dc9fb3d7281fd22085657727716f763630a4c2341460f6441c65517 WatchSource:0}: Error finding container 6f360fdf7dc9fb3d7281fd22085657727716f763630a4c2341460f6441c65517: Status 404 returned error can't find the container with id 6f360fdf7dc9fb3d7281fd22085657727716f763630a4c2341460f6441c65517
Jul 20 18:47:24.160032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:24.159999    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:24.250681 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:24.250640    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds" event={"ID":"5b428e60-10ec-430f-9ed0-9cec7a4ef272","Type":"ContainerStarted","Data":"6f360fdf7dc9fb3d7281fd22085657727716f763630a4c2341460f6441c65517"}
Jul 20 18:47:24.256050 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:24.256021    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:25.258365 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:25.258329    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds" event={"ID":"5b428e60-10ec-430f-9ed0-9cec7a4ef272","Type":"ContainerStarted","Data":"da2b5a02a5b55c84f7c6de2475a54aac024901cfa6ed0bd00ac15720aa08bdc7"}
Jul 20 18:47:25.258769 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:25.258390    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"
Jul 20 18:47:25.292357 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:25.292309    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds" podStartSLOduration=1.7455347940000001 podStartE2EDuration="2.292293766s" totalImagesPullingTime="546.758972ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:47:25.291873216 +0000 UTC m=+2042.125845604" watchObservedRunningTime="2026-07-20 18:47:25.292293766 +0000 UTC m=+2042.126266177"
Jul 20 18:47:31.270396 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:31.270363    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"
Jul 20 18:47:49.163039 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:49.163003    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bae9791c-s645z"]
Jul 20 18:47:49.166894 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:49.166876    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bae9791c-s645z"
Jul 20 18:47:49.169539 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:49.169516    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w8qkh\""
Jul 20 18:47:49.175483 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:49.175328    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bae9791c-s645z"]
Jul 20 18:47:49.226443 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:49.226409    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae89aa74-0ba5-40af-97c1-cb7d068a29c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-bae9791c-s645z\" (UID: \"ae89aa74-0ba5-40af-97c1-cb7d068a29c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bae9791c-s645z"
Jul 20 18:47:49.327767 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:49.327732    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae89aa74-0ba5-40af-97c1-cb7d068a29c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-bae9791c-s645z\" (UID: \"ae89aa74-0ba5-40af-97c1-cb7d068a29c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bae9791c-s645z"
Jul 20 18:47:49.328361 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:49.328340    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae89aa74-0ba5-40af-97c1-cb7d068a29c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-bae9791c-s645z\" (UID: \"ae89aa74-0ba5-40af-97c1-cb7d068a29c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bae9791c-s645z"
Jul 20 18:47:49.479399 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:49.479323    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bae9791c-s645z"
Jul 20 18:47:49.614082 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:49.614040    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bae9791c-s645z"]
Jul 20 18:47:49.616609 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:47:49.616584    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae89aa74_0ba5_40af_97c1_cb7d068a29c2.slice/crio-dde955064e40f44ac81778e971a315739287d4906383595b01723f0b97d0e342 WatchSource:0}: Error finding container dde955064e40f44ac81778e971a315739287d4906383595b01723f0b97d0e342: Status 404 returned error can't find the container with id dde955064e40f44ac81778e971a315739287d4906383595b01723f0b97d0e342
Jul 20 18:47:50.381415 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:50.381385    2569 generic.go:356] "Generic (PLEG): container finished" podID="ae89aa74-0ba5-40af-97c1-cb7d068a29c2" containerID="7b8ccfa877bfbeb9574a1b748f8f34d3f09c5fd1c0d92d72d745f61a6af7909e" exitCode=0
Jul 20 18:47:50.381757 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:50.381439    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bae9791c-s645z" event={"ID":"ae89aa74-0ba5-40af-97c1-cb7d068a29c2","Type":"ContainerDied","Data":"7b8ccfa877bfbeb9574a1b748f8f34d3f09c5fd1c0d92d72d745f61a6af7909e"}
Jul 20 18:47:50.381757 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:50.381490    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bae9791c-s645z" event={"ID":"ae89aa74-0ba5-40af-97c1-cb7d068a29c2","Type":"ContainerStarted","Data":"dde955064e40f44ac81778e971a315739287d4906383595b01723f0b97d0e342"}
Jul 20 18:47:51.519649 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:51.519625    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bae9791c-s645z"
Jul 20 18:47:51.548894 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:51.548868    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae89aa74-0ba5-40af-97c1-cb7d068a29c2-maas-api-service-ca\") pod \"ae89aa74-0ba5-40af-97c1-cb7d068a29c2\" (UID: \"ae89aa74-0ba5-40af-97c1-cb7d068a29c2\") "
Jul 20 18:47:51.549223 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:51.549201    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae89aa74-0ba5-40af-97c1-cb7d068a29c2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ae89aa74-0ba5-40af-97c1-cb7d068a29c2" (UID: "ae89aa74-0ba5-40af-97c1-cb7d068a29c2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:47:51.650417 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:51.650331    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae89aa74-0ba5-40af-97c1-cb7d068a29c2-maas-api-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:47:52.392534 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:52.392503    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bae9791c-s645z"
Jul 20 18:47:52.392718 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:52.392540    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bae9791c-s645z" event={"ID":"ae89aa74-0ba5-40af-97c1-cb7d068a29c2","Type":"ContainerDied","Data":"dde955064e40f44ac81778e971a315739287d4906383595b01723f0b97d0e342"}
Jul 20 18:47:52.392718 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:52.392596    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dde955064e40f44ac81778e971a315739287d4906383595b01723f0b97d0e342"
Jul 20 18:47:52.600255 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:52.600220    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:52.700179 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:52.700099    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:52.914166 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:52.914133    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:59.747831 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:59.747797    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:59.850619 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:59.850577    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:47:59.997818 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:59.997783    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"]
Jul 20 18:47:59.998377 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:47:59.998287    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds" podUID="5b428e60-10ec-430f-9ed0-9cec7a4ef272" containerName="maas-api" containerID="cri-o://da2b5a02a5b55c84f7c6de2475a54aac024901cfa6ed0bd00ac15720aa08bdc7" gracePeriod=30
Jul 20 18:48:00.026455 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.026422    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:00.247168 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.247145    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"
Jul 20 18:48:00.345117 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.344987    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rplc\" (UniqueName: \"kubernetes.io/projected/5b428e60-10ec-430f-9ed0-9cec7a4ef272-kube-api-access-6rplc\") pod \"5b428e60-10ec-430f-9ed0-9cec7a4ef272\" (UID: \"5b428e60-10ec-430f-9ed0-9cec7a4ef272\") "
Jul 20 18:48:00.345346 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.345152    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b428e60-10ec-430f-9ed0-9cec7a4ef272-openshift-service-ca\") pod \"5b428e60-10ec-430f-9ed0-9cec7a4ef272\" (UID: \"5b428e60-10ec-430f-9ed0-9cec7a4ef272\") "
Jul 20 18:48:00.345346 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.345200    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b428e60-10ec-430f-9ed0-9cec7a4ef272-maas-api-tls\") pod \"5b428e60-10ec-430f-9ed0-9cec7a4ef272\" (UID: \"5b428e60-10ec-430f-9ed0-9cec7a4ef272\") "
Jul 20 18:48:00.345898 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.345870    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b428e60-10ec-430f-9ed0-9cec7a4ef272-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5b428e60-10ec-430f-9ed0-9cec7a4ef272" (UID: "5b428e60-10ec-430f-9ed0-9cec7a4ef272"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:48:00.347656 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.347629    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b428e60-10ec-430f-9ed0-9cec7a4ef272-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5b428e60-10ec-430f-9ed0-9cec7a4ef272" (UID: "5b428e60-10ec-430f-9ed0-9cec7a4ef272"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:48:00.347955 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.347934    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b428e60-10ec-430f-9ed0-9cec7a4ef272-kube-api-access-6rplc" (OuterVolumeSpecName: "kube-api-access-6rplc") pod "5b428e60-10ec-430f-9ed0-9cec7a4ef272" (UID: "5b428e60-10ec-430f-9ed0-9cec7a4ef272"). InnerVolumeSpecName "kube-api-access-6rplc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:48:00.433485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.433453    2569 generic.go:356] "Generic (PLEG): container finished" podID="5b428e60-10ec-430f-9ed0-9cec7a4ef272" containerID="da2b5a02a5b55c84f7c6de2475a54aac024901cfa6ed0bd00ac15720aa08bdc7" exitCode=0
Jul 20 18:48:00.433690 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.433510    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"
Jul 20 18:48:00.433690 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.433531    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds" event={"ID":"5b428e60-10ec-430f-9ed0-9cec7a4ef272","Type":"ContainerDied","Data":"da2b5a02a5b55c84f7c6de2475a54aac024901cfa6ed0bd00ac15720aa08bdc7"}
Jul 20 18:48:00.433690 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.433606    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds" event={"ID":"5b428e60-10ec-430f-9ed0-9cec7a4ef272","Type":"ContainerDied","Data":"6f360fdf7dc9fb3d7281fd22085657727716f763630a4c2341460f6441c65517"}
Jul 20 18:48:00.433690 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.433627    2569 scope.go:117] "RemoveContainer" containerID="da2b5a02a5b55c84f7c6de2475a54aac024901cfa6ed0bd00ac15720aa08bdc7"
Jul 20 18:48:00.443409 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.443389    2569 scope.go:117] "RemoveContainer" containerID="da2b5a02a5b55c84f7c6de2475a54aac024901cfa6ed0bd00ac15720aa08bdc7"
Jul 20 18:48:00.443659 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:48:00.443642    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da2b5a02a5b55c84f7c6de2475a54aac024901cfa6ed0bd00ac15720aa08bdc7\": container with ID starting with da2b5a02a5b55c84f7c6de2475a54aac024901cfa6ed0bd00ac15720aa08bdc7 not found: ID does not exist" containerID="da2b5a02a5b55c84f7c6de2475a54aac024901cfa6ed0bd00ac15720aa08bdc7"
Jul 20 18:48:00.443727 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.443664    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da2b5a02a5b55c84f7c6de2475a54aac024901cfa6ed0bd00ac15720aa08bdc7"} err="failed to get container status \"da2b5a02a5b55c84f7c6de2475a54aac024901cfa6ed0bd00ac15720aa08bdc7\": rpc error: code = NotFound desc = could not find container \"da2b5a02a5b55c84f7c6de2475a54aac024901cfa6ed0bd00ac15720aa08bdc7\": container with ID starting with da2b5a02a5b55c84f7c6de2475a54aac024901cfa6ed0bd00ac15720aa08bdc7 not found: ID does not exist"
Jul 20 18:48:00.446550 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.446529    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b428e60-10ec-430f-9ed0-9cec7a4ef272-openshift-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:48:00.446550 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.446551    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b428e60-10ec-430f-9ed0-9cec7a4ef272-maas-api-tls\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:48:00.446723 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.446562    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rplc\" (UniqueName: \"kubernetes.io/projected/5b428e60-10ec-430f-9ed0-9cec7a4ef272-kube-api-access-6rplc\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:48:00.458864 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.458843    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"]
Jul 20 18:48:00.464756 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:00.464726    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bae9791c-bbd4c8968-pzzds"]
Jul 20 18:48:01.759826 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:01.759788    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b428e60-10ec-430f-9ed0-9cec7a4ef272" path="/var/lib/kubelet/pods/5b428e60-10ec-430f-9ed0-9cec7a4ef272/volumes"
Jul 20 18:48:04.257390 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:04.257347    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:04.266007 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:04.265973    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bae9791c-s645z"]
Jul 20 18:48:04.271645 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:04.271621    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bae9791c-s645z"]
Jul 20 18:48:05.510918 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:05.510882    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:05.741159 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:05.741129    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:05.759445 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:05.759414    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae89aa74-0ba5-40af-97c1-cb7d068a29c2" path="/var/lib/kubelet/pods/ae89aa74-0ba5-40af-97c1-cb7d068a29c2/volumes"
Jul 20 18:48:05.968123 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:05.968033    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:06.346248 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:06.345787    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:06.917576 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:06.917540    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:09.023314 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.023280    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:09.117718 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.117686    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"]
Jul 20 18:48:09.118153 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.118128    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae89aa74-0ba5-40af-97c1-cb7d068a29c2" containerName="revoke-keys"
Jul 20 18:48:09.118153 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.118149    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae89aa74-0ba5-40af-97c1-cb7d068a29c2" containerName="revoke-keys"
Jul 20 18:48:09.118323 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.118181    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b428e60-10ec-430f-9ed0-9cec7a4ef272" containerName="maas-api"
Jul 20 18:48:09.118323 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.118190    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b428e60-10ec-430f-9ed0-9cec7a4ef272" containerName="maas-api"
Jul 20 18:48:09.118323 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.118261    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b428e60-10ec-430f-9ed0-9cec7a4ef272" containerName="maas-api"
Jul 20 18:48:09.118323 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.118275    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae89aa74-0ba5-40af-97c1-cb7d068a29c2" containerName="revoke-keys"
Jul 20 18:48:09.120812 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.120784    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:09.122694 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.122677    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"
Jul 20 18:48:09.124964 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.124948    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-264f7364-serving-cert\""
Jul 20 18:48:09.125447 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.125428    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vvgb2\""
Jul 20 18:48:09.133475 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.133446    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"]
Jul 20 18:48:09.222576 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.222548    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:09.227888 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.227867    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/723b4537-3db1-4c6a-a0c8-3bed239d0df4-openshift-service-ca\") pod \"maas-api-e2e-derive-264f7364-5556779db-vpqjs\" (UID: \"723b4537-3db1-4c6a-a0c8-3bed239d0df4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"
Jul 20 18:48:09.227957 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.227935    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/723b4537-3db1-4c6a-a0c8-3bed239d0df4-maas-api-tls\") pod \"maas-api-e2e-derive-264f7364-5556779db-vpqjs\" (UID: \"723b4537-3db1-4c6a-a0c8-3bed239d0df4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"
Jul 20 18:48:09.227992 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.227970    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfs9k\" (UniqueName: \"kubernetes.io/projected/723b4537-3db1-4c6a-a0c8-3bed239d0df4-kube-api-access-vfs9k\") pod \"maas-api-e2e-derive-264f7364-5556779db-vpqjs\" (UID: \"723b4537-3db1-4c6a-a0c8-3bed239d0df4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"
Jul 20 18:48:09.329101 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.328985    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/723b4537-3db1-4c6a-a0c8-3bed239d0df4-openshift-service-ca\") pod \"maas-api-e2e-derive-264f7364-5556779db-vpqjs\" (UID: \"723b4537-3db1-4c6a-a0c8-3bed239d0df4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"
Jul 20 18:48:09.329268 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.329101    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/723b4537-3db1-4c6a-a0c8-3bed239d0df4-maas-api-tls\") pod \"maas-api-e2e-derive-264f7364-5556779db-vpqjs\" (UID: \"723b4537-3db1-4c6a-a0c8-3bed239d0df4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"
Jul 20 18:48:09.329268 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.329151    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfs9k\" (UniqueName: \"kubernetes.io/projected/723b4537-3db1-4c6a-a0c8-3bed239d0df4-kube-api-access-vfs9k\") pod \"maas-api-e2e-derive-264f7364-5556779db-vpqjs\" (UID: \"723b4537-3db1-4c6a-a0c8-3bed239d0df4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"
Jul 20 18:48:09.329699 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.329678    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/723b4537-3db1-4c6a-a0c8-3bed239d0df4-openshift-service-ca\") pod \"maas-api-e2e-derive-264f7364-5556779db-vpqjs\" (UID: \"723b4537-3db1-4c6a-a0c8-3bed239d0df4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"
Jul 20 18:48:09.331583 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.331563    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/723b4537-3db1-4c6a-a0c8-3bed239d0df4-maas-api-tls\") pod \"maas-api-e2e-derive-264f7364-5556779db-vpqjs\" (UID: \"723b4537-3db1-4c6a-a0c8-3bed239d0df4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"
Jul 20 18:48:09.338361 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.338341    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfs9k\" (UniqueName: \"kubernetes.io/projected/723b4537-3db1-4c6a-a0c8-3bed239d0df4-kube-api-access-vfs9k\") pod \"maas-api-e2e-derive-264f7364-5556779db-vpqjs\" (UID: \"723b4537-3db1-4c6a-a0c8-3bed239d0df4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"
Jul 20 18:48:09.342560 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.342539    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:09.434223 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.434166    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"
Jul 20 18:48:09.583972 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.583945    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"]
Jul 20 18:48:09.585853 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:48:09.585826    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod723b4537_3db1_4c6a_a0c8_3bed239d0df4.slice/crio-7078c3e383c0de117ac763c7ec3ea9e900611ab0b2ea083d4d4b49d690717ede WatchSource:0}: Error finding container 7078c3e383c0de117ac763c7ec3ea9e900611ab0b2ea083d4d4b49d690717ede: Status 404 returned error can't find the container with id 7078c3e383c0de117ac763c7ec3ea9e900611ab0b2ea083d4d4b49d690717ede
Jul 20 18:48:09.729947 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.729914    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:09.830733 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:09.830697    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:10.492230 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:10.492144    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs" event={"ID":"723b4537-3db1-4c6a-a0c8-3bed239d0df4","Type":"ContainerStarted","Data":"44eace9edcc50826760bb5c0939c79ce792c27b431df946eebce75f91bff18a8"}
Jul 20 18:48:10.492230 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:10.492205    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs" event={"ID":"723b4537-3db1-4c6a-a0c8-3bed239d0df4","Type":"ContainerStarted","Data":"7078c3e383c0de117ac763c7ec3ea9e900611ab0b2ea083d4d4b49d690717ede"}
Jul 20 18:48:10.492636 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:10.492247    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"
Jul 20 18:48:10.519260 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:10.519206    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs" podStartSLOduration=1.002551208 podStartE2EDuration="1.51918482s" totalImagesPullingTime="516.633612ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:48:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:48:10.516758245 +0000 UTC m=+2087.350730644" watchObservedRunningTime="2026-07-20 18:48:10.51918482 +0000 UTC m=+2087.353157208"
Jul 20 18:48:16.105014 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:16.104981    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:16.168964 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:16.168933    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:16.503013 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:16.502987    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"
Jul 20 18:48:30.304350 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:30.304238    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:48:30.315760 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:30.315737    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:48:33.811958 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:33.811923    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-264f7364-xvfw9"]
Jul 20 18:48:33.815398 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:33.815381    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-264f7364-xvfw9"
Jul 20 18:48:33.817988 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:33.817965    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w8qkh\""
Jul 20 18:48:33.827023 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:33.827000    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-264f7364-xvfw9"]
Jul 20 18:48:33.858917 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:33.858874    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4d53c9e-985a-492d-a3d7-aab9611eb386-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-264f7364-xvfw9\" (UID: \"e4d53c9e-985a-492d-a3d7-aab9611eb386\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-264f7364-xvfw9"
Jul 20 18:48:33.960256 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:33.960184    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4d53c9e-985a-492d-a3d7-aab9611eb386-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-264f7364-xvfw9\" (UID: \"e4d53c9e-985a-492d-a3d7-aab9611eb386\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-264f7364-xvfw9"
Jul 20 18:48:33.960836 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:33.960815    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4d53c9e-985a-492d-a3d7-aab9611eb386-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-264f7364-xvfw9\" (UID: \"e4d53c9e-985a-492d-a3d7-aab9611eb386\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-264f7364-xvfw9"
Jul 20 18:48:34.127433 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:34.127348    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-264f7364-xvfw9"
Jul 20 18:48:34.270339 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:34.270312    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-264f7364-xvfw9"]
Jul 20 18:48:34.272366 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:48:34.272335    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4d53c9e_985a_492d_a3d7_aab9611eb386.slice/crio-cb5ee77ae38f67a93e7e84a32da31e4f728e223f632ea7b55aa25dfc5ff077a8 WatchSource:0}: Error finding container cb5ee77ae38f67a93e7e84a32da31e4f728e223f632ea7b55aa25dfc5ff077a8: Status 404 returned error can't find the container with id cb5ee77ae38f67a93e7e84a32da31e4f728e223f632ea7b55aa25dfc5ff077a8
Jul 20 18:48:34.613637 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:34.613602    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-264f7364-xvfw9" event={"ID":"e4d53c9e-985a-492d-a3d7-aab9611eb386","Type":"ContainerStarted","Data":"cb5ee77ae38f67a93e7e84a32da31e4f728e223f632ea7b55aa25dfc5ff077a8"}
Jul 20 18:48:35.620722 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:35.620688    2569 generic.go:356] "Generic (PLEG): container finished" podID="e4d53c9e-985a-492d-a3d7-aab9611eb386" containerID="8c0c2f5872b20be97ffdf4793e99879cbaa458213b6d1ecf280cf179b632a987" exitCode=0
Jul 20 18:48:35.620722 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:35.620725    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-264f7364-xvfw9" event={"ID":"e4d53c9e-985a-492d-a3d7-aab9611eb386","Type":"ContainerDied","Data":"8c0c2f5872b20be97ffdf4793e99879cbaa458213b6d1ecf280cf179b632a987"}
Jul 20 18:48:36.754613 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:36.754588    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-264f7364-xvfw9"
Jul 20 18:48:36.889498 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:36.889409    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4d53c9e-985a-492d-a3d7-aab9611eb386-maas-api-service-ca\") pod \"e4d53c9e-985a-492d-a3d7-aab9611eb386\" (UID: \"e4d53c9e-985a-492d-a3d7-aab9611eb386\") "
Jul 20 18:48:36.889755 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:36.889732    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4d53c9e-985a-492d-a3d7-aab9611eb386-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e4d53c9e-985a-492d-a3d7-aab9611eb386" (UID: "e4d53c9e-985a-492d-a3d7-aab9611eb386"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:48:36.990826 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:36.990782    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4d53c9e-985a-492d-a3d7-aab9611eb386-maas-api-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:48:37.631126 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:37.631087    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-264f7364-xvfw9" event={"ID":"e4d53c9e-985a-492d-a3d7-aab9611eb386","Type":"ContainerDied","Data":"cb5ee77ae38f67a93e7e84a32da31e4f728e223f632ea7b55aa25dfc5ff077a8"}
Jul 20 18:48:37.631126 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:37.631113    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-264f7364-xvfw9"
Jul 20 18:48:37.631387 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:37.631144    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb5ee77ae38f67a93e7e84a32da31e4f728e223f632ea7b55aa25dfc5ff077a8"
Jul 20 18:48:44.350877 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.350844    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"]
Jul 20 18:48:44.351295 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.351104    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs" podUID="723b4537-3db1-4c6a-a0c8-3bed239d0df4" containerName="maas-api" containerID="cri-o://44eace9edcc50826760bb5c0939c79ce792c27b431df946eebce75f91bff18a8" gracePeriod=30
Jul 20 18:48:44.384726 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.384696    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:44.601118 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.601042    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"
Jul 20 18:48:44.668176 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.668144    2569 generic.go:356] "Generic (PLEG): container finished" podID="723b4537-3db1-4c6a-a0c8-3bed239d0df4" containerID="44eace9edcc50826760bb5c0939c79ce792c27b431df946eebce75f91bff18a8" exitCode=0
Jul 20 18:48:44.668338 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.668188    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"
Jul 20 18:48:44.668338 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.668237    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs" event={"ID":"723b4537-3db1-4c6a-a0c8-3bed239d0df4","Type":"ContainerDied","Data":"44eace9edcc50826760bb5c0939c79ce792c27b431df946eebce75f91bff18a8"}
Jul 20 18:48:44.668338 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.668285    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs" event={"ID":"723b4537-3db1-4c6a-a0c8-3bed239d0df4","Type":"ContainerDied","Data":"7078c3e383c0de117ac763c7ec3ea9e900611ab0b2ea083d4d4b49d690717ede"}
Jul 20 18:48:44.668338 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.668310    2569 scope.go:117] "RemoveContainer" containerID="44eace9edcc50826760bb5c0939c79ce792c27b431df946eebce75f91bff18a8"
Jul 20 18:48:44.679477 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.679458    2569 scope.go:117] "RemoveContainer" containerID="44eace9edcc50826760bb5c0939c79ce792c27b431df946eebce75f91bff18a8"
Jul 20 18:48:44.679761 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:48:44.679731    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44eace9edcc50826760bb5c0939c79ce792c27b431df946eebce75f91bff18a8\": container with ID starting with 44eace9edcc50826760bb5c0939c79ce792c27b431df946eebce75f91bff18a8 not found: ID does not exist" containerID="44eace9edcc50826760bb5c0939c79ce792c27b431df946eebce75f91bff18a8"
Jul 20 18:48:44.679831 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.679772    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44eace9edcc50826760bb5c0939c79ce792c27b431df946eebce75f91bff18a8"} err="failed to get container status \"44eace9edcc50826760bb5c0939c79ce792c27b431df946eebce75f91bff18a8\": rpc error: code = NotFound desc = could not find container \"44eace9edcc50826760bb5c0939c79ce792c27b431df946eebce75f91bff18a8\": container with ID starting with 44eace9edcc50826760bb5c0939c79ce792c27b431df946eebce75f91bff18a8 not found: ID does not exist"
Jul 20 18:48:44.768623 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.768593    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/723b4537-3db1-4c6a-a0c8-3bed239d0df4-maas-api-tls\") pod \"723b4537-3db1-4c6a-a0c8-3bed239d0df4\" (UID: \"723b4537-3db1-4c6a-a0c8-3bed239d0df4\") "
Jul 20 18:48:44.768787 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.768654    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfs9k\" (UniqueName: \"kubernetes.io/projected/723b4537-3db1-4c6a-a0c8-3bed239d0df4-kube-api-access-vfs9k\") pod \"723b4537-3db1-4c6a-a0c8-3bed239d0df4\" (UID: \"723b4537-3db1-4c6a-a0c8-3bed239d0df4\") "
Jul 20 18:48:44.768787 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.768674    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/723b4537-3db1-4c6a-a0c8-3bed239d0df4-openshift-service-ca\") pod \"723b4537-3db1-4c6a-a0c8-3bed239d0df4\" (UID: \"723b4537-3db1-4c6a-a0c8-3bed239d0df4\") "
Jul 20 18:48:44.769128 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.769102    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/723b4537-3db1-4c6a-a0c8-3bed239d0df4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "723b4537-3db1-4c6a-a0c8-3bed239d0df4" (UID: "723b4537-3db1-4c6a-a0c8-3bed239d0df4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:48:44.770668 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.770641    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/723b4537-3db1-4c6a-a0c8-3bed239d0df4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "723b4537-3db1-4c6a-a0c8-3bed239d0df4" (UID: "723b4537-3db1-4c6a-a0c8-3bed239d0df4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:48:44.770766 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.770707    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/723b4537-3db1-4c6a-a0c8-3bed239d0df4-kube-api-access-vfs9k" (OuterVolumeSpecName: "kube-api-access-vfs9k") pod "723b4537-3db1-4c6a-a0c8-3bed239d0df4" (UID: "723b4537-3db1-4c6a-a0c8-3bed239d0df4"). InnerVolumeSpecName "kube-api-access-vfs9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:48:44.869934 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.869855    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfs9k\" (UniqueName: \"kubernetes.io/projected/723b4537-3db1-4c6a-a0c8-3bed239d0df4-kube-api-access-vfs9k\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:48:44.869934 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.869884    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/723b4537-3db1-4c6a-a0c8-3bed239d0df4-openshift-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:48:44.869934 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.869894    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/723b4537-3db1-4c6a-a0c8-3bed239d0df4-maas-api-tls\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:48:44.995992 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:44.995962    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"]
Jul 20 18:48:45.001667 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:45.001640    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-264f7364-5556779db-vpqjs"]
Jul 20 18:48:45.759715 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:45.759682    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="723b4537-3db1-4c6a-a0c8-3bed239d0df4" path="/var/lib/kubelet/pods/723b4537-3db1-4c6a-a0c8-3bed239d0df4/volumes"
Jul 20 18:48:48.913145 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:48.913072    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-264f7364-xvfw9"]
Jul 20 18:48:48.920730 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:48.920703    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-264f7364-xvfw9"]
Jul 20 18:48:48.927080 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:48.927041    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:49.098749 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:49.098717    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:49.411681 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:49.411650    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:49.759475 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:49.759450    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4d53c9e-985a-492d-a3d7-aab9611eb386" path="/var/lib/kubelet/pods/e4d53c9e-985a-492d-a3d7-aab9611eb386/volumes"
Jul 20 18:48:49.855830 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:49.855805    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:49.969538 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:49.969501    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:51.569676 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:51.569644    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:58.791930 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:58.791891    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:58.900584 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:58.900555    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:59.041985 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:59.041904    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:59.172883 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:59.172848    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:48:59.634147 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:48:59.634116    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:49:25.580198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:49:25.580161    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:49:32.625842 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:49:32.625809    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:49:32.742070 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:49:32.742022    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:49:32.853075 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:49:32.853024    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:49:48.795256 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:49:48.795222    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:49:54.902350 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:49:54.902316    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:49:55.012872 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:49:55.012843    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:49:55.102014 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:49:55.101985    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:05.660212 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:05.660174    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:06.554145 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:06.554113    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:13.236868 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:13.236835    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:13.344357 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:13.344321    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:13.465770 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:13.465737    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:13.601000 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:13.600961    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:14.455031 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:14.455000    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:14.548189 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:14.548152    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:14.653326 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:14.653297    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:16.334955 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:16.334921    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:23.189297 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:23.189260    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:25.134112 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:25.134077    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:30.774214 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:30.774169    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:30.875336 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:30.875301    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:31.014157 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:31.014124    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:31.544186 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:31.544152    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:32.420236 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:32.420202    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:34.310218 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.310184    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:34.408116 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.408085    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:34.525588 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.525552    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"]
Jul 20 18:50:34.525954 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.525942    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="723b4537-3db1-4c6a-a0c8-3bed239d0df4" containerName="maas-api"
Jul 20 18:50:34.526002 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.525955    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="723b4537-3db1-4c6a-a0c8-3bed239d0df4" containerName="maas-api"
Jul 20 18:50:34.526002 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.525963    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4d53c9e-985a-492d-a3d7-aab9611eb386" containerName="revoke-keys"
Jul 20 18:50:34.526002 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.525969    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4d53c9e-985a-492d-a3d7-aab9611eb386" containerName="revoke-keys"
Jul 20 18:50:34.526301 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.526080    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4d53c9e-985a-492d-a3d7-aab9611eb386" containerName="revoke-keys"
Jul 20 18:50:34.526301 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.526093    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="723b4537-3db1-4c6a-a0c8-3bed239d0df4" containerName="maas-api"
Jul 20 18:50:34.529182 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.529164    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"
Jul 20 18:50:34.531764 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.531744    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-bffbe307-serving-cert\""
Jul 20 18:50:34.531956 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.531937    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vvgb2\""
Jul 20 18:50:34.540404 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.540358    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"]
Jul 20 18:50:34.566214 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.566148    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:34.691168 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.691144    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-openshift-service-ca\") pod \"maas-api-e2e-mt-bffbe307-77889464c-xlwp9\" (UID: \"6d2c1c33-4e32-404b-9499-b7bc9dc244bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"
Jul 20 18:50:34.691338 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.691214    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh4zt\" (UniqueName: \"kubernetes.io/projected/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-kube-api-access-gh4zt\") pod \"maas-api-e2e-mt-bffbe307-77889464c-xlwp9\" (UID: \"6d2c1c33-4e32-404b-9499-b7bc9dc244bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"
Jul 20 18:50:34.691338 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.691269    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-maas-api-tls\") pod \"maas-api-e2e-mt-bffbe307-77889464c-xlwp9\" (UID: \"6d2c1c33-4e32-404b-9499-b7bc9dc244bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"
Jul 20 18:50:34.768407 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.768380    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:34.792602 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.792572    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-maas-api-tls\") pod \"maas-api-e2e-mt-bffbe307-77889464c-xlwp9\" (UID: \"6d2c1c33-4e32-404b-9499-b7bc9dc244bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"
Jul 20 18:50:34.792749 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.792619    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-openshift-service-ca\") pod \"maas-api-e2e-mt-bffbe307-77889464c-xlwp9\" (UID: \"6d2c1c33-4e32-404b-9499-b7bc9dc244bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"
Jul 20 18:50:34.792749 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.792675    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh4zt\" (UniqueName: \"kubernetes.io/projected/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-kube-api-access-gh4zt\") pod \"maas-api-e2e-mt-bffbe307-77889464c-xlwp9\" (UID: \"6d2c1c33-4e32-404b-9499-b7bc9dc244bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"
Jul 20 18:50:34.793373 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.793346    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-openshift-service-ca\") pod \"maas-api-e2e-mt-bffbe307-77889464c-xlwp9\" (UID: \"6d2c1c33-4e32-404b-9499-b7bc9dc244bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"
Jul 20 18:50:34.795020 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.795000    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-maas-api-tls\") pod \"maas-api-e2e-mt-bffbe307-77889464c-xlwp9\" (UID: \"6d2c1c33-4e32-404b-9499-b7bc9dc244bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"
Jul 20 18:50:34.801198 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.801175    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh4zt\" (UniqueName: \"kubernetes.io/projected/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-kube-api-access-gh4zt\") pod \"maas-api-e2e-mt-bffbe307-77889464c-xlwp9\" (UID: \"6d2c1c33-4e32-404b-9499-b7bc9dc244bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"
Jul 20 18:50:34.840234 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.840176    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"
Jul 20 18:50:34.983674 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:34.983642    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"]
Jul 20 18:50:34.987190 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:50:34.987159    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d2c1c33_4e32_404b_9499_b7bc9dc244bd.slice/crio-da3c6b78d84b301b720db03f9fcce13cc75573cd7fca4559a77666613b613c26 WatchSource:0}: Error finding container da3c6b78d84b301b720db03f9fcce13cc75573cd7fca4559a77666613b613c26: Status 404 returned error can't find the container with id da3c6b78d84b301b720db03f9fcce13cc75573cd7fca4559a77666613b613c26
Jul 20 18:50:35.136658 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:35.136565    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:35.230500 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:35.230461    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9" event={"ID":"6d2c1c33-4e32-404b-9499-b7bc9dc244bd","Type":"ContainerStarted","Data":"da3c6b78d84b301b720db03f9fcce13cc75573cd7fca4559a77666613b613c26"}
Jul 20 18:50:35.235526 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:35.235500    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:36.237623 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:36.237587    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9" event={"ID":"6d2c1c33-4e32-404b-9499-b7bc9dc244bd","Type":"ContainerStarted","Data":"b47fe6d0689c49de0b788a8d5cdab81e1c534f8f6ad1f22961c3a1c72eea78a8"}
Jul 20 18:50:36.238035 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:36.237660    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"
Jul 20 18:50:36.257675 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:36.257621    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9" podStartSLOduration=1.759059074 podStartE2EDuration="2.257602709s" totalImagesPullingTime="498.543635ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:50:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:50:36.257540283 +0000 UTC m=+2233.091512694" watchObservedRunningTime="2026-07-20 18:50:36.257602709 +0000 UTC m=+2233.091575098"
Jul 20 18:50:38.742991 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:38.742958    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:39.435182 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:39.435148    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:39.538949 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:39.538913    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:39.868613 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:39.868580    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:42.247695 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:42.247666    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"
Jul 20 18:50:45.736912 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:45.736834    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:45.838233 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:45.838201    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:46.124236 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:46.124202    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6"]
Jul 20 18:50:46.128680 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:46.128661    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6"
Jul 20 18:50:46.133590 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:46.133561    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w8qkh\""
Jul 20 18:50:46.139576 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:46.139550    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6"]
Jul 20 18:50:46.303143 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:46.303109    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dbffa81-6b89-430f-8f59-63521fbecc47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6\" (UID: \"4dbffa81-6b89-430f-8f59-63521fbecc47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6"
Jul 20 18:50:46.404549 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:46.404458    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dbffa81-6b89-430f-8f59-63521fbecc47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6\" (UID: \"4dbffa81-6b89-430f-8f59-63521fbecc47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6"
Jul 20 18:50:46.405158 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:46.405131    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dbffa81-6b89-430f-8f59-63521fbecc47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6\" (UID: \"4dbffa81-6b89-430f-8f59-63521fbecc47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6"
Jul 20 18:50:46.440506 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:46.440474    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6"
Jul 20 18:50:46.576427 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:46.576402    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6"]
Jul 20 18:50:46.578436 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:50:46.578413    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dbffa81_6b89_430f_8f59_63521fbecc47.slice/crio-9521ed239e468025c9187729e8cb97f50bec58906cf7ee485c392ca00d8c85e9 WatchSource:0}: Error finding container 9521ed239e468025c9187729e8cb97f50bec58906cf7ee485c392ca00d8c85e9: Status 404 returned error can't find the container with id 9521ed239e468025c9187729e8cb97f50bec58906cf7ee485c392ca00d8c85e9
Jul 20 18:50:47.293466 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:47.293435    2569 generic.go:356] "Generic (PLEG): container finished" podID="4dbffa81-6b89-430f-8f59-63521fbecc47" containerID="87ffcf2474a549b2d1fcecdbceaa24ab90b1e40e6acf0d54c85073f66279c770" exitCode=0
Jul 20 18:50:47.293883 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:47.293496    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6" event={"ID":"4dbffa81-6b89-430f-8f59-63521fbecc47","Type":"ContainerDied","Data":"87ffcf2474a549b2d1fcecdbceaa24ab90b1e40e6acf0d54c85073f66279c770"}
Jul 20 18:50:47.293883 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:47.293533    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6" event={"ID":"4dbffa81-6b89-430f-8f59-63521fbecc47","Type":"ContainerStarted","Data":"9521ed239e468025c9187729e8cb97f50bec58906cf7ee485c392ca00d8c85e9"}
Jul 20 18:50:48.432455 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:48.432431    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6"
Jul 20 18:50:48.625897 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:48.625806    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dbffa81-6b89-430f-8f59-63521fbecc47-maas-api-service-ca\") pod \"4dbffa81-6b89-430f-8f59-63521fbecc47\" (UID: \"4dbffa81-6b89-430f-8f59-63521fbecc47\") "
Jul 20 18:50:48.626219 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:48.626194    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dbffa81-6b89-430f-8f59-63521fbecc47-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4dbffa81-6b89-430f-8f59-63521fbecc47" (UID: "4dbffa81-6b89-430f-8f59-63521fbecc47"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:50:48.727283 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:48.727229    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dbffa81-6b89-430f-8f59-63521fbecc47-maas-api-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:50:49.305506 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:49.305472    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6" event={"ID":"4dbffa81-6b89-430f-8f59-63521fbecc47","Type":"ContainerDied","Data":"9521ed239e468025c9187729e8cb97f50bec58906cf7ee485c392ca00d8c85e9"}
Jul 20 18:50:49.305506 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:49.305499    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6"
Jul 20 18:50:49.305721 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:49.305518    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9521ed239e468025c9187729e8cb97f50bec58906cf7ee485c392ca00d8c85e9"
Jul 20 18:50:56.642493 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:56.642457    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"]
Jul 20 18:50:56.642960 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:56.642667    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9" podUID="6d2c1c33-4e32-404b-9499-b7bc9dc244bd" containerName="maas-api" containerID="cri-o://b47fe6d0689c49de0b788a8d5cdab81e1c534f8f6ad1f22961c3a1c72eea78a8" gracePeriod=30
Jul 20 18:50:56.679589 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:56.679555    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:50:56.891699 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:56.891669    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"
Jul 20 18:50:57.003236 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.003205    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-openshift-service-ca\") pod \"6d2c1c33-4e32-404b-9499-b7bc9dc244bd\" (UID: \"6d2c1c33-4e32-404b-9499-b7bc9dc244bd\") "
Jul 20 18:50:57.003429 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.003305    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-maas-api-tls\") pod \"6d2c1c33-4e32-404b-9499-b7bc9dc244bd\" (UID: \"6d2c1c33-4e32-404b-9499-b7bc9dc244bd\") "
Jul 20 18:50:57.003429 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.003347    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gh4zt\" (UniqueName: \"kubernetes.io/projected/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-kube-api-access-gh4zt\") pod \"6d2c1c33-4e32-404b-9499-b7bc9dc244bd\" (UID: \"6d2c1c33-4e32-404b-9499-b7bc9dc244bd\") "
Jul 20 18:50:57.003652 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.003593    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6d2c1c33-4e32-404b-9499-b7bc9dc244bd" (UID: "6d2c1c33-4e32-404b-9499-b7bc9dc244bd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:50:57.003746 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.003718    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-openshift-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:50:57.005379 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.005346    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6d2c1c33-4e32-404b-9499-b7bc9dc244bd" (UID: "6d2c1c33-4e32-404b-9499-b7bc9dc244bd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:50:57.005478 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.005424    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-kube-api-access-gh4zt" (OuterVolumeSpecName: "kube-api-access-gh4zt") pod "6d2c1c33-4e32-404b-9499-b7bc9dc244bd" (UID: "6d2c1c33-4e32-404b-9499-b7bc9dc244bd"). InnerVolumeSpecName "kube-api-access-gh4zt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:50:57.104860 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.104812    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-maas-api-tls\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:50:57.104860 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.104855    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gh4zt\" (UniqueName: \"kubernetes.io/projected/6d2c1c33-4e32-404b-9499-b7bc9dc244bd-kube-api-access-gh4zt\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:50:57.350185 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.350088    2569 generic.go:356] "Generic (PLEG): container finished" podID="6d2c1c33-4e32-404b-9499-b7bc9dc244bd" containerID="b47fe6d0689c49de0b788a8d5cdab81e1c534f8f6ad1f22961c3a1c72eea78a8" exitCode=0
Jul 20 18:50:57.350185 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.350125    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"
Jul 20 18:50:57.350185 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.350156    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9" event={"ID":"6d2c1c33-4e32-404b-9499-b7bc9dc244bd","Type":"ContainerDied","Data":"b47fe6d0689c49de0b788a8d5cdab81e1c534f8f6ad1f22961c3a1c72eea78a8"}
Jul 20 18:50:57.350432 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.350207    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9" event={"ID":"6d2c1c33-4e32-404b-9499-b7bc9dc244bd","Type":"ContainerDied","Data":"da3c6b78d84b301b720db03f9fcce13cc75573cd7fca4559a77666613b613c26"}
Jul 20 18:50:57.350432 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.350235    2569 scope.go:117] "RemoveContainer" containerID="b47fe6d0689c49de0b788a8d5cdab81e1c534f8f6ad1f22961c3a1c72eea78a8"
Jul 20 18:50:57.359410 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.359390    2569 scope.go:117] "RemoveContainer" containerID="b47fe6d0689c49de0b788a8d5cdab81e1c534f8f6ad1f22961c3a1c72eea78a8"
Jul 20 18:50:57.359662 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:50:57.359641    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b47fe6d0689c49de0b788a8d5cdab81e1c534f8f6ad1f22961c3a1c72eea78a8\": container with ID starting with b47fe6d0689c49de0b788a8d5cdab81e1c534f8f6ad1f22961c3a1c72eea78a8 not found: ID does not exist" containerID="b47fe6d0689c49de0b788a8d5cdab81e1c534f8f6ad1f22961c3a1c72eea78a8"
Jul 20 18:50:57.359716 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.359670    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b47fe6d0689c49de0b788a8d5cdab81e1c534f8f6ad1f22961c3a1c72eea78a8"} err="failed to get container status \"b47fe6d0689c49de0b788a8d5cdab81e1c534f8f6ad1f22961c3a1c72eea78a8\": rpc error: code = NotFound desc = could not find container \"b47fe6d0689c49de0b788a8d5cdab81e1c534f8f6ad1f22961c3a1c72eea78a8\": container with ID starting with b47fe6d0689c49de0b788a8d5cdab81e1c534f8f6ad1f22961c3a1c72eea78a8 not found: ID does not exist"
Jul 20 18:50:57.372260 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.372232    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"]
Jul 20 18:50:57.376726 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.376705    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bffbe307-77889464c-xlwp9"]
Jul 20 18:50:57.759505 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:50:57.759473    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d2c1c33-4e32-404b-9499-b7bc9dc244bd" path="/var/lib/kubelet/pods/6d2c1c33-4e32-404b-9499-b7bc9dc244bd/volumes"
Jul 20 18:51:01.211882 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:01.211843    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6"]
Jul 20 18:51:01.214732 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:01.214708    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bffbe307-fg5w6"]
Jul 20 18:51:01.218192 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:01.218161    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:01.595309 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:01.595276    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:01.759330 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:01.759298    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dbffa81-6b89-430f-8f59-63521fbecc47" path="/var/lib/kubelet/pods/4dbffa81-6b89-430f-8f59-63521fbecc47/volumes"
Jul 20 18:51:07.523363 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:07.523324    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:07.623641 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:07.623609    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:07.810098 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:07.810004    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:08.024589 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:08.024552    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:08.225379 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:08.225285    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:08.434662 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:08.434632    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:08.533545 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:08.533511    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:14.357324 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:14.357280    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:14.431432 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:14.431371    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:14.547264 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:14.547231    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:14.694693 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:14.694600    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:15.478204 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:15.478175    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:21.168299 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:21.168269    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:21.262325 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:21.262295    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:28.026111 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:28.026016    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:28.399823 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:28.399727    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:28.594321 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:28.594283    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:31.514953 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.514922    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"]
Jul 20 18:51:31.515543 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.515522    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dbffa81-6b89-430f-8f59-63521fbecc47" containerName="revoke-keys"
Jul 20 18:51:31.515630 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.515546    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dbffa81-6b89-430f-8f59-63521fbecc47" containerName="revoke-keys"
Jul 20 18:51:31.515630 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.515577    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d2c1c33-4e32-404b-9499-b7bc9dc244bd" containerName="maas-api"
Jul 20 18:51:31.515630 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.515586    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d2c1c33-4e32-404b-9499-b7bc9dc244bd" containerName="maas-api"
Jul 20 18:51:31.515784 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.515679    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d2c1c33-4e32-404b-9499-b7bc9dc244bd" containerName="maas-api"
Jul 20 18:51:31.515784 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.515691    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dbffa81-6b89-430f-8f59-63521fbecc47" containerName="revoke-keys"
Jul 20 18:51:31.518972 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.518953    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"
Jul 20 18:51:31.521669 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.521649    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d9481028-serving-cert\""
Jul 20 18:51:31.521779 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.521670    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vvgb2\""
Jul 20 18:51:31.530627 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.530591    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x77ln\" (UniqueName: \"kubernetes.io/projected/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-kube-api-access-x77ln\") pod \"maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts\" (UID: \"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"
Jul 20 18:51:31.530748 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.530659    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-maas-api-tls\") pod \"maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts\" (UID: \"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"
Jul 20 18:51:31.530748 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.530728    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-openshift-service-ca\") pod \"maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts\" (UID: \"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"
Jul 20 18:51:31.532357 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.532336    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"]
Jul 20 18:51:31.632106 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.632050    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x77ln\" (UniqueName: \"kubernetes.io/projected/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-kube-api-access-x77ln\") pod \"maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts\" (UID: \"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"
Jul 20 18:51:31.632281 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.632126    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-maas-api-tls\") pod \"maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts\" (UID: \"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"
Jul 20 18:51:31.632281 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.632170    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-openshift-service-ca\") pod \"maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts\" (UID: \"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"
Jul 20 18:51:31.632281 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:51:31.632262    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-serving-cert: secret "maas-api-e2e-mt-d9481028-serving-cert" not found
Jul 20 18:51:31.632384 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:51:31.632333    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-maas-api-tls podName:bf0824d5-cf13-4c4a-8e26-eb04fdf7b694 nodeName:}" failed. No retries permitted until 2026-07-20 18:51:32.132317407 +0000 UTC m=+2288.966289776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-maas-api-tls") pod "maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts" (UID: "bf0824d5-cf13-4c4a-8e26-eb04fdf7b694") : secret "maas-api-e2e-mt-d9481028-serving-cert" not found
Jul 20 18:51:31.632769 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.632753    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-openshift-service-ca\") pod \"maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts\" (UID: \"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"
Jul 20 18:51:31.642773 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:31.642749    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x77ln\" (UniqueName: \"kubernetes.io/projected/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-kube-api-access-x77ln\") pod \"maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts\" (UID: \"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"
Jul 20 18:51:32.136982 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:32.136946    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-maas-api-tls\") pod \"maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts\" (UID: \"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"
Jul 20 18:51:32.139377 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:32.139350    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-maas-api-tls\") pod \"maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts\" (UID: \"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"
Jul 20 18:51:32.149698 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:32.149674    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:32.262217 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:32.262184    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:32.412090 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:32.411996    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:32.429371 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:32.429339    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"
Jul 20 18:51:32.513027 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:32.512994    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:32.566592 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:32.566521    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"]
Jul 20 18:51:32.568952 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:51:32.568928    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf0824d5_cf13_4c4a_8e26_eb04fdf7b694.slice/crio-65d3be1c3eb3e31546099a1fc2df45e7568e406565945da018c090bd9358bdc7 WatchSource:0}: Error finding container 65d3be1c3eb3e31546099a1fc2df45e7568e406565945da018c090bd9358bdc7: Status 404 returned error can't find the container with id 65d3be1c3eb3e31546099a1fc2df45e7568e406565945da018c090bd9358bdc7
Jul 20 18:51:32.570318 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:32.570301    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 18:51:33.547654 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:33.547617    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts" event={"ID":"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694","Type":"ContainerStarted","Data":"0a80ca661703fa77bd6c29c9a33fd07a606ccd379ed3dc419afdd297ec1ef6b9"}
Jul 20 18:51:33.547903 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:33.547677    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts" event={"ID":"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694","Type":"ContainerStarted","Data":"65d3be1c3eb3e31546099a1fc2df45e7568e406565945da018c090bd9358bdc7"}
Jul 20 18:51:33.547903 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:33.547784    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"
Jul 20 18:51:33.566961 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:33.566916    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts" podStartSLOduration=2.032927424 podStartE2EDuration="2.566901998s" totalImagesPullingTime="533.974574ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:51:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:51:33.565492872 +0000 UTC m=+2290.399465260" watchObservedRunningTime="2026-07-20 18:51:33.566901998 +0000 UTC m=+2290.400874386"
Jul 20 18:51:35.879113 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:35.879042    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:39.558954 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:39.558929    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"
Jul 20 18:51:45.724653 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:45.724619    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:51:56.375382 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:51:56.375346    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:52:19.563118 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:19.562941    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8"]
Jul 20 18:52:19.566594 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:19.566575    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8"
Jul 20 18:52:19.569126 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:19.569103    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w8qkh\""
Jul 20 18:52:19.611182 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:19.611145    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8"]
Jul 20 18:52:19.678214 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:19.678175    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e264f4c5-d596-4ea2-a85f-a2c30123f260-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d9481028-dqth8\" (UID: \"e264f4c5-d596-4ea2-a85f-a2c30123f260\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8"
Jul 20 18:52:19.779766 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:19.779734    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e264f4c5-d596-4ea2-a85f-a2c30123f260-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d9481028-dqth8\" (UID: \"e264f4c5-d596-4ea2-a85f-a2c30123f260\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8"
Jul 20 18:52:19.780376 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:19.780357    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e264f4c5-d596-4ea2-a85f-a2c30123f260-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d9481028-dqth8\" (UID: \"e264f4c5-d596-4ea2-a85f-a2c30123f260\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8"
Jul 20 18:52:19.877304 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:19.877223    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8"
Jul 20 18:52:20.018178 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:20.018083    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8"]
Jul 20 18:52:20.020854 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:52:20.020818    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode264f4c5_d596_4ea2_a85f_a2c30123f260.slice/crio-4c3d47c606b72427ad5811ebfa53486b0aff088b750c9bfe65cae69402400efb WatchSource:0}: Error finding container 4c3d47c606b72427ad5811ebfa53486b0aff088b750c9bfe65cae69402400efb: Status 404 returned error can't find the container with id 4c3d47c606b72427ad5811ebfa53486b0aff088b750c9bfe65cae69402400efb
Jul 20 18:52:20.595441 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:20.595353    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:52:20.698147 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:20.698110    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:52:20.798139 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:20.798100    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8" event={"ID":"e264f4c5-d596-4ea2-a85f-a2c30123f260","Type":"ContainerStarted","Data":"860647bdaecdc5ae29b51ee7935fbd76255f1a360403af7cc1f095dc9f14467d"}
Jul 20 18:52:20.798350 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:20.798187    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8" event={"ID":"e264f4c5-d596-4ea2-a85f-a2c30123f260","Type":"ContainerStarted","Data":"4c3d47c606b72427ad5811ebfa53486b0aff088b750c9bfe65cae69402400efb"}
Jul 20 18:52:20.816296 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:20.816234    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8" podStartSLOduration=1.816217635 podStartE2EDuration="1.816217635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:52:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:52:20.814495818 +0000 UTC m=+2337.648468208" watchObservedRunningTime="2026-07-20 18:52:20.816217635 +0000 UTC m=+2337.650190023"
Jul 20 18:52:21.803847 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:21.803813    2569 generic.go:356] "Generic (PLEG): container finished" podID="e264f4c5-d596-4ea2-a85f-a2c30123f260" containerID="860647bdaecdc5ae29b51ee7935fbd76255f1a360403af7cc1f095dc9f14467d" exitCode=0
Jul 20 18:52:21.804291 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:21.803869    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8" event={"ID":"e264f4c5-d596-4ea2-a85f-a2c30123f260","Type":"ContainerDied","Data":"860647bdaecdc5ae29b51ee7935fbd76255f1a360403af7cc1f095dc9f14467d"}
Jul 20 18:52:22.946865 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:22.946843    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8"
Jul 20 18:52:23.015102 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:23.015070    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e264f4c5-d596-4ea2-a85f-a2c30123f260-maas-api-service-ca\") pod \"e264f4c5-d596-4ea2-a85f-a2c30123f260\" (UID: \"e264f4c5-d596-4ea2-a85f-a2c30123f260\") "
Jul 20 18:52:23.015345 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:23.015322    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e264f4c5-d596-4ea2-a85f-a2c30123f260-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e264f4c5-d596-4ea2-a85f-a2c30123f260" (UID: "e264f4c5-d596-4ea2-a85f-a2c30123f260"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:52:23.116090 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:23.115985    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e264f4c5-d596-4ea2-a85f-a2c30123f260-maas-api-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:52:23.816394 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:23.816367    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8"
Jul 20 18:52:23.816574 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:23.816405    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8" event={"ID":"e264f4c5-d596-4ea2-a85f-a2c30123f260","Type":"ContainerDied","Data":"4c3d47c606b72427ad5811ebfa53486b0aff088b750c9bfe65cae69402400efb"}
Jul 20 18:52:23.816574 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:23.816474    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c3d47c606b72427ad5811ebfa53486b0aff088b750c9bfe65cae69402400efb"
Jul 20 18:52:23.910775 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:23.910750    2569 scope.go:117] "RemoveContainer" containerID="f1e676fddd2c7d6154227c19c2c44d0df86eaba90203abe437519ae9fca315a9"
Jul 20 18:52:35.121942 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.121904    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"]
Jul 20 18:52:35.122556 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.122219    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts" podUID="bf0824d5-cf13-4c4a-8e26-eb04fdf7b694" containerName="maas-api" containerID="cri-o://0a80ca661703fa77bd6c29c9a33fd07a606ccd379ed3dc419afdd297ec1ef6b9" gracePeriod=30
Jul 20 18:52:35.261780 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.261747    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:52:35.369352 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.369328    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"
Jul 20 18:52:35.538747 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.538706    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x77ln\" (UniqueName: \"kubernetes.io/projected/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-kube-api-access-x77ln\") pod \"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694\" (UID: \"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694\") "
Jul 20 18:52:35.538942 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.538793    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-openshift-service-ca\") pod \"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694\" (UID: \"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694\") "
Jul 20 18:52:35.538942 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.538913    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-maas-api-tls\") pod \"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694\" (UID: \"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694\") "
Jul 20 18:52:35.539180 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.539156    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bf0824d5-cf13-4c4a-8e26-eb04fdf7b694" (UID: "bf0824d5-cf13-4c4a-8e26-eb04fdf7b694"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:52:35.540887 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.540860    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bf0824d5-cf13-4c4a-8e26-eb04fdf7b694" (UID: "bf0824d5-cf13-4c4a-8e26-eb04fdf7b694"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:52:35.540986 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.540931    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-kube-api-access-x77ln" (OuterVolumeSpecName: "kube-api-access-x77ln") pod "bf0824d5-cf13-4c4a-8e26-eb04fdf7b694" (UID: "bf0824d5-cf13-4c4a-8e26-eb04fdf7b694"). InnerVolumeSpecName "kube-api-access-x77ln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:52:35.640346 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.640307    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-maas-api-tls\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:52:35.640346 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.640338    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x77ln\" (UniqueName: \"kubernetes.io/projected/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-kube-api-access-x77ln\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:52:35.640346 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.640348    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694-openshift-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:52:35.885973 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.885887    2569 generic.go:356] "Generic (PLEG): container finished" podID="bf0824d5-cf13-4c4a-8e26-eb04fdf7b694" containerID="0a80ca661703fa77bd6c29c9a33fd07a606ccd379ed3dc419afdd297ec1ef6b9" exitCode=0
Jul 20 18:52:35.885973 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.885936    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"
Jul 20 18:52:35.885973 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.885964    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts" event={"ID":"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694","Type":"ContainerDied","Data":"0a80ca661703fa77bd6c29c9a33fd07a606ccd379ed3dc419afdd297ec1ef6b9"}
Jul 20 18:52:35.886242 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.886009    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts" event={"ID":"bf0824d5-cf13-4c4a-8e26-eb04fdf7b694","Type":"ContainerDied","Data":"65d3be1c3eb3e31546099a1fc2df45e7568e406565945da018c090bd9358bdc7"}
Jul 20 18:52:35.886242 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.886034    2569 scope.go:117] "RemoveContainer" containerID="0a80ca661703fa77bd6c29c9a33fd07a606ccd379ed3dc419afdd297ec1ef6b9"
Jul 20 18:52:35.894609 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.894589    2569 scope.go:117] "RemoveContainer" containerID="0a80ca661703fa77bd6c29c9a33fd07a606ccd379ed3dc419afdd297ec1ef6b9"
Jul 20 18:52:35.894862 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:52:35.894842    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a80ca661703fa77bd6c29c9a33fd07a606ccd379ed3dc419afdd297ec1ef6b9\": container with ID starting with 0a80ca661703fa77bd6c29c9a33fd07a606ccd379ed3dc419afdd297ec1ef6b9 not found: ID does not exist" containerID="0a80ca661703fa77bd6c29c9a33fd07a606ccd379ed3dc419afdd297ec1ef6b9"
Jul 20 18:52:35.894930 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.894870    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a80ca661703fa77bd6c29c9a33fd07a606ccd379ed3dc419afdd297ec1ef6b9"} err="failed to get container status \"0a80ca661703fa77bd6c29c9a33fd07a606ccd379ed3dc419afdd297ec1ef6b9\": rpc error: code = NotFound desc = could not find container \"0a80ca661703fa77bd6c29c9a33fd07a606ccd379ed3dc419afdd297ec1ef6b9\": container with ID starting with 0a80ca661703fa77bd6c29c9a33fd07a606ccd379ed3dc419afdd297ec1ef6b9 not found: ID does not exist"
Jul 20 18:52:35.904759 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.904737    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"]
Jul 20 18:52:35.909648 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:35.909623    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d9481028-7bbfc87d86-v6xts"]
Jul 20 18:52:37.759719 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:37.759683    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf0824d5-cf13-4c4a-8e26-eb04fdf7b694" path="/var/lib/kubelet/pods/bf0824d5-cf13-4c4a-8e26-eb04fdf7b694/volumes"
Jul 20 18:52:39.644244 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:39.644212    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:52:39.648925 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:39.648895    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8"]
Jul 20 18:52:39.652418 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:39.652393    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d9481028-dqth8"]
Jul 20 18:52:39.758715 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:39.758683    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e264f4c5-d596-4ea2-a85f-a2c30123f260" path="/var/lib/kubelet/pods/e264f4c5-d596-4ea2-a85f-a2c30123f260/volumes"
Jul 20 18:52:41.875903 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:41.875864    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:52:46.913596 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:46.913555    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:52:53.783991 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:53.783960    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:52:53.884346 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:52:53.884316    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:23.155872 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:23.155809    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:23.932358 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:23.932326    2569 scope.go:117] "RemoveContainer" containerID="0b92c342b02ce646e4fcdd55ecdb6972ac7897627ead48671e5c9da4eb52db87"
Jul 20 18:53:23.963284 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:23.963248    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:29.766301 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:29.766261    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:29.877483 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:29.877450    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:30.101674 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:30.101564    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:30.339101 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:30.338984    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:53:30.350664 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:30.350642    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:53:30.398360 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:30.398275    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:31.458805 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:31.458772    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:31.566431 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:31.566395    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:32.497629 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:32.497594    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:32.656534 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:32.656499    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:34.215810 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:34.215760    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:40.484657 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:40.484612    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:40.630650 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:40.630611    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:41.086073 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:41.086020    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:47.223153 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:47.223039    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:47.295462 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:47.295431    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:47.383044 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:47.383009    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:57.925201 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:57.925166    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:53:58.336773 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:53:58.336735    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:19.241259 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:19.241225    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:23.957262 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:23.957230    2569 scope.go:117] "RemoveContainer" containerID="7b8ccfa877bfbeb9574a1b748f8f34d3f09c5fd1c0d92d72d745f61a6af7909e"
Jul 20 18:54:25.317601 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:25.317565    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:25.418382 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:25.418345    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:25.735916 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:25.735838    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:26.037549 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:26.037483    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:26.131500 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:26.131451    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:29.130871 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.130828    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:29.230853 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.230821    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:29.303718 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.303683    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"]
Jul 20 18:54:29.304127 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.304113    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e264f4c5-d596-4ea2-a85f-a2c30123f260" containerName="revoke-keys"
Jul 20 18:54:29.304177 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.304129    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e264f4c5-d596-4ea2-a85f-a2c30123f260" containerName="revoke-keys"
Jul 20 18:54:29.304177 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.304156    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf0824d5-cf13-4c4a-8e26-eb04fdf7b694" containerName="maas-api"
Jul 20 18:54:29.304177 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.304162    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf0824d5-cf13-4c4a-8e26-eb04fdf7b694" containerName="maas-api"
Jul 20 18:54:29.304275 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.304227    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf0824d5-cf13-4c4a-8e26-eb04fdf7b694" containerName="maas-api"
Jul 20 18:54:29.304275 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.304235    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e264f4c5-d596-4ea2-a85f-a2c30123f260" containerName="revoke-keys"
Jul 20 18:54:29.308956 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.308933    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"
Jul 20 18:54:29.311370 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.311345    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-c017f1-serving-cert\""
Jul 20 18:54:29.311485 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.311438    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vvgb2\""
Jul 20 18:54:29.319634 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.319613    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"]
Jul 20 18:54:29.338463 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.338435    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:29.415219 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.415132    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgpkm\" (UniqueName: \"kubernetes.io/projected/52e219e3-db01-4d71-913b-54c339fd9b5d-kube-api-access-tgpkm\") pod \"maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8\" (UID: \"52e219e3-db01-4d71-913b-54c339fd9b5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"
Jul 20 18:54:29.415219 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.415214    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52e219e3-db01-4d71-913b-54c339fd9b5d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8\" (UID: \"52e219e3-db01-4d71-913b-54c339fd9b5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"
Jul 20 18:54:29.415423 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.415266    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52e219e3-db01-4d71-913b-54c339fd9b5d-maas-api-tls\") pod \"maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8\" (UID: \"52e219e3-db01-4d71-913b-54c339fd9b5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"
Jul 20 18:54:29.515907 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.515869    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52e219e3-db01-4d71-913b-54c339fd9b5d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8\" (UID: \"52e219e3-db01-4d71-913b-54c339fd9b5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"
Jul 20 18:54:29.516109 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.515925    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52e219e3-db01-4d71-913b-54c339fd9b5d-maas-api-tls\") pod \"maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8\" (UID: \"52e219e3-db01-4d71-913b-54c339fd9b5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"
Jul 20 18:54:29.516109 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.515968    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgpkm\" (UniqueName: \"kubernetes.io/projected/52e219e3-db01-4d71-913b-54c339fd9b5d-kube-api-access-tgpkm\") pod \"maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8\" (UID: \"52e219e3-db01-4d71-913b-54c339fd9b5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"
Jul 20 18:54:29.516525 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.516496    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52e219e3-db01-4d71-913b-54c339fd9b5d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8\" (UID: \"52e219e3-db01-4d71-913b-54c339fd9b5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"
Jul 20 18:54:29.518340 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.518317    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52e219e3-db01-4d71-913b-54c339fd9b5d-maas-api-tls\") pod \"maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8\" (UID: \"52e219e3-db01-4d71-913b-54c339fd9b5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"
Jul 20 18:54:29.524537 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.524518    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgpkm\" (UniqueName: \"kubernetes.io/projected/52e219e3-db01-4d71-913b-54c339fd9b5d-kube-api-access-tgpkm\") pod \"maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8\" (UID: \"52e219e3-db01-4d71-913b-54c339fd9b5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"
Jul 20 18:54:29.532820 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.532778    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:29.620927 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.620890    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"
Jul 20 18:54:29.763472 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.763445    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"]
Jul 20 18:54:29.765296 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:54:29.765257    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52e219e3_db01_4d71_913b_54c339fd9b5d.slice/crio-4b8d11d9f865fdf9a2c565f99d088a4f1b63c6efbfacf95ff83fc221221cd93d WatchSource:0}: Error finding container 4b8d11d9f865fdf9a2c565f99d088a4f1b63c6efbfacf95ff83fc221221cd93d: Status 404 returned error can't find the container with id 4b8d11d9f865fdf9a2c565f99d088a4f1b63c6efbfacf95ff83fc221221cd93d
Jul 20 18:54:29.970007 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:29.969917    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:30.070156 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:30.070119    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:30.486228 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:30.486191    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8" event={"ID":"52e219e3-db01-4d71-913b-54c339fd9b5d","Type":"ContainerStarted","Data":"d8e99e961c3c096e88be8fa2653ca9da79c5e56be85e2a41ca8ce9df039f3fa0"}
Jul 20 18:54:30.486689 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:30.486248    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8" event={"ID":"52e219e3-db01-4d71-913b-54c339fd9b5d","Type":"ContainerStarted","Data":"4b8d11d9f865fdf9a2c565f99d088a4f1b63c6efbfacf95ff83fc221221cd93d"}
Jul 20 18:54:30.486689 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:30.486276    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"
Jul 20 18:54:30.505011 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:30.504959    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8" podStartSLOduration=1.122303051 podStartE2EDuration="1.504943655s" totalImagesPullingTime="382.640604ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:54:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:54:30.503375897 +0000 UTC m=+2467.337348286" watchObservedRunningTime="2026-07-20 18:54:30.504943655 +0000 UTC m=+2467.338916043"
Jul 20 18:54:33.680285 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:33.680235    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:34.009589 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:34.009539    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:34.422284 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:34.422203    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:34.569233 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:34.569201    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:36.497191 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:36.497162    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"
Jul 20 18:54:37.383221 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:37.383189    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:37.449269 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:37.449225    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"]
Jul 20 18:54:37.453279 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:37.453257    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"
Jul 20 18:54:37.455784 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:37.455762    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-9e16a1-serving-cert\""
Jul 20 18:54:37.463008 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:37.462982    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"]
Jul 20 18:54:37.467217 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:37.467195    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:37.490331 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:37.490305    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41892e6f-9d83-4596-8519-013dca0c6143-maas-api-tls\") pod \"maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6\" (UID: \"41892e6f-9d83-4596-8519-013dca0c6143\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"
Jul 20 18:54:37.490467 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:37.490338    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwkk6\" (UniqueName: \"kubernetes.io/projected/41892e6f-9d83-4596-8519-013dca0c6143-kube-api-access-gwkk6\") pod \"maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6\" (UID: \"41892e6f-9d83-4596-8519-013dca0c6143\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"
Jul 20 18:54:37.490467 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:37.490360    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41892e6f-9d83-4596-8519-013dca0c6143-openshift-service-ca\") pod \"maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6\" (UID: \"41892e6f-9d83-4596-8519-013dca0c6143\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"
Jul 20 18:54:37.528587 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:37.528553    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:37.590967 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:37.590931    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41892e6f-9d83-4596-8519-013dca0c6143-maas-api-tls\") pod \"maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6\" (UID: \"41892e6f-9d83-4596-8519-013dca0c6143\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"
Jul 20 18:54:37.591152 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:37.590971    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwkk6\" (UniqueName: \"kubernetes.io/projected/41892e6f-9d83-4596-8519-013dca0c6143-kube-api-access-gwkk6\") pod \"maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6\" (UID: \"41892e6f-9d83-4596-8519-013dca0c6143\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"
Jul 20 18:54:37.591152 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:54:37.591134    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-serving-cert: secret "maas-api-e2e-inf-b-9e16a1-serving-cert" not found
Jul 20 18:54:37.591226 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:54:37.591199    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41892e6f-9d83-4596-8519-013dca0c6143-maas-api-tls podName:41892e6f-9d83-4596-8519-013dca0c6143 nodeName:}" failed. No retries permitted until 2026-07-20 18:54:38.091184016 +0000 UTC m=+2474.925156383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/41892e6f-9d83-4596-8519-013dca0c6143-maas-api-tls") pod "maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6" (UID: "41892e6f-9d83-4596-8519-013dca0c6143") : secret "maas-api-e2e-inf-b-9e16a1-serving-cert" not found
Jul 20 18:54:37.591226 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:37.591225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41892e6f-9d83-4596-8519-013dca0c6143-openshift-service-ca\") pod \"maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6\" (UID: \"41892e6f-9d83-4596-8519-013dca0c6143\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"
Jul 20 18:54:37.591817 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:37.591799    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41892e6f-9d83-4596-8519-013dca0c6143-openshift-service-ca\") pod \"maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6\" (UID: \"41892e6f-9d83-4596-8519-013dca0c6143\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"
Jul 20 18:54:37.602209 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:37.602180    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwkk6\" (UniqueName: \"kubernetes.io/projected/41892e6f-9d83-4596-8519-013dca0c6143-kube-api-access-gwkk6\") pod \"maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6\" (UID: \"41892e6f-9d83-4596-8519-013dca0c6143\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"
Jul 20 18:54:37.646964 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:37.646883    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:38.069073 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:38.069015    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:38.095362 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:38.095330    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41892e6f-9d83-4596-8519-013dca0c6143-maas-api-tls\") pod \"maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6\" (UID: \"41892e6f-9d83-4596-8519-013dca0c6143\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"
Jul 20 18:54:38.097829 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:38.097805    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41892e6f-9d83-4596-8519-013dca0c6143-maas-api-tls\") pod \"maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6\" (UID: \"41892e6f-9d83-4596-8519-013dca0c6143\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"
Jul 20 18:54:38.168028 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:38.167990    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:38.366733 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:38.366648    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"
Jul 20 18:54:38.503436 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:38.503406    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"]
Jul 20 18:54:38.505226 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:54:38.505200    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41892e6f_9d83_4596_8519_013dca0c6143.slice/crio-2309a4ccfecac4d67849ea32fff0e014c5237895d6b3f77462648eeea455a776 WatchSource:0}: Error finding container 2309a4ccfecac4d67849ea32fff0e014c5237895d6b3f77462648eeea455a776: Status 404 returned error can't find the container with id 2309a4ccfecac4d67849ea32fff0e014c5237895d6b3f77462648eeea455a776
Jul 20 18:54:38.527647 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:38.527616    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6" event={"ID":"41892e6f-9d83-4596-8519-013dca0c6143","Type":"ContainerStarted","Data":"2309a4ccfecac4d67849ea32fff0e014c5237895d6b3f77462648eeea455a776"}
Jul 20 18:54:39.533734 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:39.533704    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6" event={"ID":"41892e6f-9d83-4596-8519-013dca0c6143","Type":"ContainerStarted","Data":"3a68d268d359c34048b9260d9166f70551e3a0af36443336bf82e3d6044312d3"}
Jul 20 18:54:39.534208 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:39.533827    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"
Jul 20 18:54:39.554279 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:39.554230    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6" podStartSLOduration=2.187723722 podStartE2EDuration="2.554212675s" totalImagesPullingTime="366.488953ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:54:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:54:39.553018143 +0000 UTC m=+2476.386990531" watchObservedRunningTime="2026-07-20 18:54:39.554212675 +0000 UTC m=+2476.388185065"
Jul 20 18:54:41.889616 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:41.889581    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:45.544990 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:45.544958    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"
Jul 20 18:54:51.830328 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:51.830295    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:51.934780 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:51.934749    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:58.817855 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:58.817814    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:59.896360 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:59.896327    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:54:59.996601 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:54:59.996562    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:55:21.161180 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:21.161089    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:55:22.793187 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.793146    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"]
Jul 20 18:55:22.796868 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.796853    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.800467 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.800440    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9e16a1\"/\"openshift-service-ca.crt\""
Jul 20 18:55:22.800600 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.800440    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9e16a1\"/\"default-dockercfg-j86j8\""
Jul 20 18:55:22.800600 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.800441    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9e16a1\"/\"test-model-9e16a1-kserve-self-signed-certs\""
Jul 20 18:55:22.800600 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.800447    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9e16a1\"/\"kube-root-ca.crt\""
Jul 20 18:55:22.806297 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.806272    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"]
Jul 20 18:55:22.810375 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.810353    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04832233-eee9-448a-b424-be0c4bc54654-tls-certs\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.810502 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.810399    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf26z\" (UniqueName: \"kubernetes.io/projected/04832233-eee9-448a-b424-be0c4bc54654-kube-api-access-xf26z\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.810502 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.810457    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-home\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.810502 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.810474    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-model-cache\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.810502 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.810494    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-dshm\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.810688 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.810571    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-tmp-dir\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.815376 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.815350    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:55:22.912041 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.912006    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-tmp-dir\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.912234 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.912112    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04832233-eee9-448a-b424-be0c4bc54654-tls-certs\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.912234 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.912151    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf26z\" (UniqueName: \"kubernetes.io/projected/04832233-eee9-448a-b424-be0c4bc54654-kube-api-access-xf26z\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.912234 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.912200    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-home\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.912234 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.912214    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-model-cache\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.912234 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.912235    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-dshm\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.912480 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.912449    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-tmp-dir\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.912569 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.912548    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-home\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.912635 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.912615    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-model-cache\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.914409 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.914389    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-dshm\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.914690 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.914673    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04832233-eee9-448a-b424-be0c4bc54654-tls-certs\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:22.923404 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:22.923379    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf26z\" (UniqueName: \"kubernetes.io/projected/04832233-eee9-448a-b424-be0c4bc54654-kube-api-access-xf26z\") pod \"test-model-9e16a1-kserve-7f4dfb7d59-5bz5c\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") " pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:23.107391 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:23.107293    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:23.239928 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:23.239900    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"]
Jul 20 18:55:23.241646 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:55:23.241615    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04832233_eee9_448a_b424_be0c4bc54654.slice/crio-fa875e222d1cce828372947fa454f34bbd3ae2183b3d138cf272b15d3ce79c26 WatchSource:0}: Error finding container fa875e222d1cce828372947fa454f34bbd3ae2183b3d138cf272b15d3ce79c26: Status 404 returned error can't find the container with id fa875e222d1cce828372947fa454f34bbd3ae2183b3d138cf272b15d3ce79c26
Jul 20 18:55:23.763311 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:23.763280    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c" event={"ID":"04832233-eee9-448a-b424-be0c4bc54654","Type":"ContainerStarted","Data":"abcfc8ef6f41e8c6dadca28a07d6261a77e8deae79ebfb2afa1523efac602a1f"}
Jul 20 18:55:23.763486 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:23.763319    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c" event={"ID":"04832233-eee9-448a-b424-be0c4bc54654","Type":"ContainerStarted","Data":"fa875e222d1cce828372947fa454f34bbd3ae2183b3d138cf272b15d3ce79c26"}
Jul 20 18:55:23.780995 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:23.780939    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c" podStartSLOduration=1.7809248229999999 podStartE2EDuration="1.780924823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:55:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:55:23.779774414 +0000 UTC m=+2520.613746824" watchObservedRunningTime="2026-07-20 18:55:23.780924823 +0000 UTC m=+2520.614897212"
Jul 20 18:55:23.976956 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:23.976923    2569 scope.go:117] "RemoveContainer" containerID="8c0c2f5872b20be97ffdf4793e99879cbaa458213b6d1ecf280cf179b632a987"
Jul 20 18:55:33.107432 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:33.107393    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:33.108088 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:33.107473    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:33.119842 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:33.119819    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:55:33.825102 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:55:33.825072    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:56:11.979251 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:11.979216    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:12.899684 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:12.899649    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"]
Jul 20 18:56:14.129938 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:14.129903    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:16.557271 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:16.557231    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:16.658535 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:16.658498    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:23.569725 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:23.569678    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"]
Jul 20 18:56:23.570204 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:23.569918    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8" podUID="52e219e3-db01-4d71-913b-54c339fd9b5d" containerName="maas-api" containerID="cri-o://d8e99e961c3c096e88be8fa2653ca9da79c5e56be85e2a41ca8ce9df039f3fa0" gracePeriod=30
Jul 20 18:56:23.599305 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:23.599273    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:24.018219 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.018195    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"
Jul 20 18:56:24.082093 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.082036    2569 generic.go:356] "Generic (PLEG): container finished" podID="52e219e3-db01-4d71-913b-54c339fd9b5d" containerID="d8e99e961c3c096e88be8fa2653ca9da79c5e56be85e2a41ca8ce9df039f3fa0" exitCode=0
Jul 20 18:56:24.082259 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.082111    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"
Jul 20 18:56:24.082259 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.082126    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8" event={"ID":"52e219e3-db01-4d71-913b-54c339fd9b5d","Type":"ContainerDied","Data":"d8e99e961c3c096e88be8fa2653ca9da79c5e56be85e2a41ca8ce9df039f3fa0"}
Jul 20 18:56:24.082259 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.082202    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8" event={"ID":"52e219e3-db01-4d71-913b-54c339fd9b5d","Type":"ContainerDied","Data":"4b8d11d9f865fdf9a2c565f99d088a4f1b63c6efbfacf95ff83fc221221cd93d"}
Jul 20 18:56:24.082259 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.082226    2569 scope.go:117] "RemoveContainer" containerID="d8e99e961c3c096e88be8fa2653ca9da79c5e56be85e2a41ca8ce9df039f3fa0"
Jul 20 18:56:24.090935 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.090913    2569 scope.go:117] "RemoveContainer" containerID="d8e99e961c3c096e88be8fa2653ca9da79c5e56be85e2a41ca8ce9df039f3fa0"
Jul 20 18:56:24.091278 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:56:24.091244    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8e99e961c3c096e88be8fa2653ca9da79c5e56be85e2a41ca8ce9df039f3fa0\": container with ID starting with d8e99e961c3c096e88be8fa2653ca9da79c5e56be85e2a41ca8ce9df039f3fa0 not found: ID does not exist" containerID="d8e99e961c3c096e88be8fa2653ca9da79c5e56be85e2a41ca8ce9df039f3fa0"
Jul 20 18:56:24.091334 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.091294    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8e99e961c3c096e88be8fa2653ca9da79c5e56be85e2a41ca8ce9df039f3fa0"} err="failed to get container status \"d8e99e961c3c096e88be8fa2653ca9da79c5e56be85e2a41ca8ce9df039f3fa0\": rpc error: code = NotFound desc = could not find container \"d8e99e961c3c096e88be8fa2653ca9da79c5e56be85e2a41ca8ce9df039f3fa0\": container with ID starting with d8e99e961c3c096e88be8fa2653ca9da79c5e56be85e2a41ca8ce9df039f3fa0 not found: ID does not exist"
Jul 20 18:56:24.102751 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.102697    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgpkm\" (UniqueName: \"kubernetes.io/projected/52e219e3-db01-4d71-913b-54c339fd9b5d-kube-api-access-tgpkm\") pod \"52e219e3-db01-4d71-913b-54c339fd9b5d\" (UID: \"52e219e3-db01-4d71-913b-54c339fd9b5d\") "
Jul 20 18:56:24.102751 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.102737    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52e219e3-db01-4d71-913b-54c339fd9b5d-openshift-service-ca\") pod \"52e219e3-db01-4d71-913b-54c339fd9b5d\" (UID: \"52e219e3-db01-4d71-913b-54c339fd9b5d\") "
Jul 20 18:56:24.102891 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.102757    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52e219e3-db01-4d71-913b-54c339fd9b5d-maas-api-tls\") pod \"52e219e3-db01-4d71-913b-54c339fd9b5d\" (UID: \"52e219e3-db01-4d71-913b-54c339fd9b5d\") "
Jul 20 18:56:24.103032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.103014    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52e219e3-db01-4d71-913b-54c339fd9b5d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "52e219e3-db01-4d71-913b-54c339fd9b5d" (UID: "52e219e3-db01-4d71-913b-54c339fd9b5d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:56:24.104804 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.104776    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52e219e3-db01-4d71-913b-54c339fd9b5d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "52e219e3-db01-4d71-913b-54c339fd9b5d" (UID: "52e219e3-db01-4d71-913b-54c339fd9b5d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:56:24.104887 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.104851    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52e219e3-db01-4d71-913b-54c339fd9b5d-kube-api-access-tgpkm" (OuterVolumeSpecName: "kube-api-access-tgpkm") pod "52e219e3-db01-4d71-913b-54c339fd9b5d" (UID: "52e219e3-db01-4d71-913b-54c339fd9b5d"). InnerVolumeSpecName "kube-api-access-tgpkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:56:24.203781 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.203741    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgpkm\" (UniqueName: \"kubernetes.io/projected/52e219e3-db01-4d71-913b-54c339fd9b5d-kube-api-access-tgpkm\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:56:24.203781 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.203780    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52e219e3-db01-4d71-913b-54c339fd9b5d-openshift-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:56:24.203781 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.203791    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52e219e3-db01-4d71-913b-54c339fd9b5d-maas-api-tls\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:56:24.403850 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.403819    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"]
Jul 20 18:56:24.407093 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:24.407067    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c017f1-5466dcc54d-jvgx8"]
Jul 20 18:56:25.759911 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:25.759870    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52e219e3-db01-4d71-913b-54c339fd9b5d" path="/var/lib/kubelet/pods/52e219e3-db01-4d71-913b-54c339fd9b5d/volumes"
Jul 20 18:56:27.920023 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:27.919987    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c" podUID="04832233-eee9-448a-b424-be0c4bc54654" containerName="main" containerID="cri-o://abcfc8ef6f41e8c6dadca28a07d6261a77e8deae79ebfb2afa1523efac602a1f" gracePeriod=45
Jul 20 18:56:28.106491 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.106458    2569 generic.go:356] "Generic (PLEG): container finished" podID="04832233-eee9-448a-b424-be0c4bc54654" containerID="abcfc8ef6f41e8c6dadca28a07d6261a77e8deae79ebfb2afa1523efac602a1f" exitCode=0
Jul 20 18:56:28.106680 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.106493    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c" event={"ID":"04832233-eee9-448a-b424-be0c4bc54654","Type":"ContainerDied","Data":"abcfc8ef6f41e8c6dadca28a07d6261a77e8deae79ebfb2afa1523efac602a1f"}
Jul 20 18:56:28.132110 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.132076    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:28.174355 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.174293    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:56:28.343207 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.343167    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xf26z\" (UniqueName: \"kubernetes.io/projected/04832233-eee9-448a-b424-be0c4bc54654-kube-api-access-xf26z\") pod \"04832233-eee9-448a-b424-be0c4bc54654\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") "
Jul 20 18:56:28.343368 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.343233    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04832233-eee9-448a-b424-be0c4bc54654-tls-certs\") pod \"04832233-eee9-448a-b424-be0c4bc54654\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") "
Jul 20 18:56:28.343368 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.343288    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-home\") pod \"04832233-eee9-448a-b424-be0c4bc54654\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") "
Jul 20 18:56:28.343368 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.343310    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-tmp-dir\") pod \"04832233-eee9-448a-b424-be0c4bc54654\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") "
Jul 20 18:56:28.343368 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.343333    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-model-cache\") pod \"04832233-eee9-448a-b424-be0c4bc54654\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") "
Jul 20 18:56:28.343368 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.343355    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-dshm\") pod \"04832233-eee9-448a-b424-be0c4bc54654\" (UID: \"04832233-eee9-448a-b424-be0c4bc54654\") "
Jul 20 18:56:28.343657 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.343533    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-model-cache" (OuterVolumeSpecName: "model-cache") pod "04832233-eee9-448a-b424-be0c4bc54654" (UID: "04832233-eee9-448a-b424-be0c4bc54654"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 18:56:28.343657 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.343558    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-home" (OuterVolumeSpecName: "home") pod "04832233-eee9-448a-b424-be0c4bc54654" (UID: "04832233-eee9-448a-b424-be0c4bc54654"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 18:56:28.343657 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.343598    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "04832233-eee9-448a-b424-be0c4bc54654" (UID: "04832233-eee9-448a-b424-be0c4bc54654"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 18:56:28.343777 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.343685    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-model-cache\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:56:28.345525 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.345496    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-dshm" (OuterVolumeSpecName: "dshm") pod "04832233-eee9-448a-b424-be0c4bc54654" (UID: "04832233-eee9-448a-b424-be0c4bc54654"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 18:56:28.345525 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.345516    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04832233-eee9-448a-b424-be0c4bc54654-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "04832233-eee9-448a-b424-be0c4bc54654" (UID: "04832233-eee9-448a-b424-be0c4bc54654"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:56:28.345647 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.345558    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04832233-eee9-448a-b424-be0c4bc54654-kube-api-access-xf26z" (OuterVolumeSpecName: "kube-api-access-xf26z") pod "04832233-eee9-448a-b424-be0c4bc54654" (UID: "04832233-eee9-448a-b424-be0c4bc54654"). InnerVolumeSpecName "kube-api-access-xf26z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:56:28.444539 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.444458    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-home\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:56:28.444539 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.444485    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-tmp-dir\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:56:28.444539 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.444493    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04832233-eee9-448a-b424-be0c4bc54654-dshm\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:56:28.444539 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.444502    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xf26z\" (UniqueName: \"kubernetes.io/projected/04832233-eee9-448a-b424-be0c4bc54654-kube-api-access-xf26z\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:56:28.444539 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:28.444512    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04832233-eee9-448a-b424-be0c4bc54654-tls-certs\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:56:29.101182 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:29.101141    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:29.112527 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:29.112504    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"
Jul 20 18:56:29.112663 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:29.112528    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c" event={"ID":"04832233-eee9-448a-b424-be0c4bc54654","Type":"ContainerDied","Data":"fa875e222d1cce828372947fa454f34bbd3ae2183b3d138cf272b15d3ce79c26"}
Jul 20 18:56:29.112663 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:29.112577    2569 scope.go:117] "RemoveContainer" containerID="abcfc8ef6f41e8c6dadca28a07d6261a77e8deae79ebfb2afa1523efac602a1f"
Jul 20 18:56:29.134253 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:29.134230    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"]
Jul 20 18:56:29.140770 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:29.140746    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-9e16a1/test-model-9e16a1-kserve-7f4dfb7d59-5bz5c"]
Jul 20 18:56:29.759416 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:29.759383    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04832233-eee9-448a-b424-be0c4bc54654" path="/var/lib/kubelet/pods/04832233-eee9-448a-b424-be0c4bc54654/volumes"
Jul 20 18:56:34.918224 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:34.918191    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:35.017101 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.017068    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:35.364548 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.364514    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:35.474167 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.474132    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:35.590527 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.590485    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz"]
Jul 20 18:56:35.590966 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.590948    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04832233-eee9-448a-b424-be0c4bc54654" containerName="main"
Jul 20 18:56:35.591078 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.590968    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="04832233-eee9-448a-b424-be0c4bc54654" containerName="main"
Jul 20 18:56:35.591078 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.590985    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52e219e3-db01-4d71-913b-54c339fd9b5d" containerName="maas-api"
Jul 20 18:56:35.591078 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.590993    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="52e219e3-db01-4d71-913b-54c339fd9b5d" containerName="maas-api"
Jul 20 18:56:35.591238 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.591124    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="04832233-eee9-448a-b424-be0c4bc54654" containerName="main"
Jul 20 18:56:35.591238 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.591140    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="52e219e3-db01-4d71-913b-54c339fd9b5d" containerName="maas-api"
Jul 20 18:56:35.594362 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.594343    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz"
Jul 20 18:56:35.596524 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.596504    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w8qkh\""
Jul 20 18:56:35.602094 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.601843    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz"]
Jul 20 18:56:35.712018 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.711934    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/179a40ee-917f-4b92-923f-6529a1ef1eae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz\" (UID: \"179a40ee-917f-4b92-923f-6529a1ef1eae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz"
Jul 20 18:56:35.812885 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.812851    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/179a40ee-917f-4b92-923f-6529a1ef1eae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz\" (UID: \"179a40ee-917f-4b92-923f-6529a1ef1eae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz"
Jul 20 18:56:35.813504 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.813485    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/179a40ee-917f-4b92-923f-6529a1ef1eae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz\" (UID: \"179a40ee-917f-4b92-923f-6529a1ef1eae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz"
Jul 20 18:56:35.906260 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:35.906215    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz"
Jul 20 18:56:36.044582 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:36.044555    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz"]
Jul 20 18:56:36.046693 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:56:36.046658    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod179a40ee_917f_4b92_923f_6529a1ef1eae.slice/crio-8dce90ad96e8a46ecd3e168570a394053a61118eb0a3ebe541197b6abd3c9fdb WatchSource:0}: Error finding container 8dce90ad96e8a46ecd3e168570a394053a61118eb0a3ebe541197b6abd3c9fdb: Status 404 returned error can't find the container with id 8dce90ad96e8a46ecd3e168570a394053a61118eb0a3ebe541197b6abd3c9fdb
Jul 20 18:56:36.149602 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:36.149558    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz" event={"ID":"179a40ee-917f-4b92-923f-6529a1ef1eae","Type":"ContainerStarted","Data":"8dce90ad96e8a46ecd3e168570a394053a61118eb0a3ebe541197b6abd3c9fdb"}
Jul 20 18:56:36.252674 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:36.252649    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:36.355852 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:36.355815    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:37.156281 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:37.156189    2569 generic.go:356] "Generic (PLEG): container finished" podID="179a40ee-917f-4b92-923f-6529a1ef1eae" containerID="1c29aaa3262bc7773749922533ce90f122ef53d0337dbfd1785433cb3d667d0c" exitCode=0
Jul 20 18:56:37.156698 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:37.156276    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz" event={"ID":"179a40ee-917f-4b92-923f-6529a1ef1eae","Type":"ContainerDied","Data":"1c29aaa3262bc7773749922533ce90f122ef53d0337dbfd1785433cb3d667d0c"}
Jul 20 18:56:38.296632 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:38.296608    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz"
Jul 20 18:56:38.439210 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:38.439132    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/179a40ee-917f-4b92-923f-6529a1ef1eae-maas-api-service-ca\") pod \"179a40ee-917f-4b92-923f-6529a1ef1eae\" (UID: \"179a40ee-917f-4b92-923f-6529a1ef1eae\") "
Jul 20 18:56:38.439481 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:38.439460    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/179a40ee-917f-4b92-923f-6529a1ef1eae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "179a40ee-917f-4b92-923f-6529a1ef1eae" (UID: "179a40ee-917f-4b92-923f-6529a1ef1eae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:56:38.540633 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:38.540594    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/179a40ee-917f-4b92-923f-6529a1ef1eae-maas-api-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:56:39.167025 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:39.166993    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz"
Jul 20 18:56:39.167228 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:39.167027    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz" event={"ID":"179a40ee-917f-4b92-923f-6529a1ef1eae","Type":"ContainerDied","Data":"8dce90ad96e8a46ecd3e168570a394053a61118eb0a3ebe541197b6abd3c9fdb"}
Jul 20 18:56:39.167228 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:39.167105    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8dce90ad96e8a46ecd3e168570a394053a61118eb0a3ebe541197b6abd3c9fdb"
Jul 20 18:56:46.030225 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:46.030147    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"]
Jul 20 18:56:46.030757 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:46.030381    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6" podUID="41892e6f-9d83-4596-8519-013dca0c6143" containerName="maas-api" containerID="cri-o://3a68d268d359c34048b9260d9166f70551e3a0af36443336bf82e3d6044312d3" gracePeriod=30
Jul 20 18:56:46.080050 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:46.080014    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:46.474525 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:46.474498    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"
Jul 20 18:56:46.495708 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:46.495681    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwkk6\" (UniqueName: \"kubernetes.io/projected/41892e6f-9d83-4596-8519-013dca0c6143-kube-api-access-gwkk6\") pod \"41892e6f-9d83-4596-8519-013dca0c6143\" (UID: \"41892e6f-9d83-4596-8519-013dca0c6143\") "
Jul 20 18:56:46.495859 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:46.495727    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41892e6f-9d83-4596-8519-013dca0c6143-maas-api-tls\") pod \"41892e6f-9d83-4596-8519-013dca0c6143\" (UID: \"41892e6f-9d83-4596-8519-013dca0c6143\") "
Jul 20 18:56:46.495925 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:46.495881    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41892e6f-9d83-4596-8519-013dca0c6143-openshift-service-ca\") pod \"41892e6f-9d83-4596-8519-013dca0c6143\" (UID: \"41892e6f-9d83-4596-8519-013dca0c6143\") "
Jul 20 18:56:46.496213 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:46.496192    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41892e6f-9d83-4596-8519-013dca0c6143-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "41892e6f-9d83-4596-8519-013dca0c6143" (UID: "41892e6f-9d83-4596-8519-013dca0c6143"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:56:46.496314 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:46.496294    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41892e6f-9d83-4596-8519-013dca0c6143-openshift-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:56:46.497786 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:46.497756    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41892e6f-9d83-4596-8519-013dca0c6143-kube-api-access-gwkk6" (OuterVolumeSpecName: "kube-api-access-gwkk6") pod "41892e6f-9d83-4596-8519-013dca0c6143" (UID: "41892e6f-9d83-4596-8519-013dca0c6143"). InnerVolumeSpecName "kube-api-access-gwkk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:56:46.497895 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:46.497877    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41892e6f-9d83-4596-8519-013dca0c6143-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "41892e6f-9d83-4596-8519-013dca0c6143" (UID: "41892e6f-9d83-4596-8519-013dca0c6143"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:56:46.596833 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:46.596744    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwkk6\" (UniqueName: \"kubernetes.io/projected/41892e6f-9d83-4596-8519-013dca0c6143-kube-api-access-gwkk6\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:56:46.596833 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:46.596777    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41892e6f-9d83-4596-8519-013dca0c6143-maas-api-tls\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:56:47.207541 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:47.207510    2569 generic.go:356] "Generic (PLEG): container finished" podID="41892e6f-9d83-4596-8519-013dca0c6143" containerID="3a68d268d359c34048b9260d9166f70551e3a0af36443336bf82e3d6044312d3" exitCode=0
Jul 20 18:56:47.207541 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:47.207550    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"
Jul 20 18:56:47.208000 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:47.207588    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6" event={"ID":"41892e6f-9d83-4596-8519-013dca0c6143","Type":"ContainerDied","Data":"3a68d268d359c34048b9260d9166f70551e3a0af36443336bf82e3d6044312d3"}
Jul 20 18:56:47.208000 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:47.207645    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6" event={"ID":"41892e6f-9d83-4596-8519-013dca0c6143","Type":"ContainerDied","Data":"2309a4ccfecac4d67849ea32fff0e014c5237895d6b3f77462648eeea455a776"}
Jul 20 18:56:47.208000 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:47.207669    2569 scope.go:117] "RemoveContainer" containerID="3a68d268d359c34048b9260d9166f70551e3a0af36443336bf82e3d6044312d3"
Jul 20 18:56:47.221750 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:47.221717    2569 scope.go:117] "RemoveContainer" containerID="3a68d268d359c34048b9260d9166f70551e3a0af36443336bf82e3d6044312d3"
Jul 20 18:56:47.222114 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:56:47.222084    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a68d268d359c34048b9260d9166f70551e3a0af36443336bf82e3d6044312d3\": container with ID starting with 3a68d268d359c34048b9260d9166f70551e3a0af36443336bf82e3d6044312d3 not found: ID does not exist" containerID="3a68d268d359c34048b9260d9166f70551e3a0af36443336bf82e3d6044312d3"
Jul 20 18:56:47.222227 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:47.222126    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a68d268d359c34048b9260d9166f70551e3a0af36443336bf82e3d6044312d3"} err="failed to get container status \"3a68d268d359c34048b9260d9166f70551e3a0af36443336bf82e3d6044312d3\": rpc error: code = NotFound desc = could not find container \"3a68d268d359c34048b9260d9166f70551e3a0af36443336bf82e3d6044312d3\": container with ID starting with 3a68d268d359c34048b9260d9166f70551e3a0af36443336bf82e3d6044312d3 not found: ID does not exist"
Jul 20 18:56:47.235274 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:47.235245    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"]
Jul 20 18:56:47.240959 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:47.240935    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9e16a1-f8b4d6766-gpgp6"]
Jul 20 18:56:47.759263 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:47.759231    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41892e6f-9d83-4596-8519-013dca0c6143" path="/var/lib/kubelet/pods/41892e6f-9d83-4596-8519-013dca0c6143/volumes"
Jul 20 18:56:50.672551 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:50.672516    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:50.673007 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:50.672523    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz"]
Jul 20 18:56:50.674577 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:50.674544    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9e16a1-gv5wz"]
Jul 20 18:56:51.053342 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:51.053307    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:51.761292 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:51.761257    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="179a40ee-917f-4b92-923f-6529a1ef1eae" path="/var/lib/kubelet/pods/179a40ee-917f-4b92-923f-6529a1ef1eae/volumes"
Jul 20 18:56:56.798280 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:56.798247    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:56.898712 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:56.898677    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:57.077189 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:57.077076    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:57.299889 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:57.299854    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:57.647022 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:57.646984    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:57.966190 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:57.966107    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:56:58.198114 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:56:58.198081    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:00.793929 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:00.793887    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:00.897226 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:00.897185    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"]
Jul 20 18:57:00.897845 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:00.897826    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41892e6f-9d83-4596-8519-013dca0c6143" containerName="maas-api"
Jul 20 18:57:00.897845 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:00.897846    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="41892e6f-9d83-4596-8519-013dca0c6143" containerName="maas-api"
Jul 20 18:57:00.897969 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:00.897877    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="179a40ee-917f-4b92-923f-6529a1ef1eae" containerName="revoke-keys"
Jul 20 18:57:00.897969 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:00.897887    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="179a40ee-917f-4b92-923f-6529a1ef1eae" containerName="revoke-keys"
Jul 20 18:57:00.898038 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:00.897983    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="179a40ee-917f-4b92-923f-6529a1ef1eae" containerName="revoke-keys"
Jul 20 18:57:00.898038 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:00.897998    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="41892e6f-9d83-4596-8519-013dca0c6143" containerName="maas-api"
Jul 20 18:57:00.901232 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:00.901215    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"
Jul 20 18:57:00.903324 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:00.903298    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:00.903546 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:00.903528    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-9eed89-serving-cert\""
Jul 20 18:57:00.903629 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:00.903557    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vvgb2\""
Jul 20 18:57:00.914019 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:00.913993    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"]
Jul 20 18:57:00.916411 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:00.916386    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98498b18-74b8-4610-88b0-f36a9ffc8c05-maas-api-tls\") pod \"maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k\" (UID: \"98498b18-74b8-4610-88b0-f36a9ffc8c05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"
Jul 20 18:57:00.916522 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:00.916458    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98498b18-74b8-4610-88b0-f36a9ffc8c05-openshift-service-ca\") pod \"maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k\" (UID: \"98498b18-74b8-4610-88b0-f36a9ffc8c05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"
Jul 20 18:57:00.916522 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:00.916501    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh2wk\" (UniqueName: \"kubernetes.io/projected/98498b18-74b8-4610-88b0-f36a9ffc8c05-kube-api-access-wh2wk\") pod \"maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k\" (UID: \"98498b18-74b8-4610-88b0-f36a9ffc8c05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"
Jul 20 18:57:01.002257 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:01.002221    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:01.017618 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:01.017588    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98498b18-74b8-4610-88b0-f36a9ffc8c05-maas-api-tls\") pod \"maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k\" (UID: \"98498b18-74b8-4610-88b0-f36a9ffc8c05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"
Jul 20 18:57:01.017732 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:01.017656    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98498b18-74b8-4610-88b0-f36a9ffc8c05-openshift-service-ca\") pod \"maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k\" (UID: \"98498b18-74b8-4610-88b0-f36a9ffc8c05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"
Jul 20 18:57:01.017732 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:01.017693    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh2wk\" (UniqueName: \"kubernetes.io/projected/98498b18-74b8-4610-88b0-f36a9ffc8c05-kube-api-access-wh2wk\") pod \"maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k\" (UID: \"98498b18-74b8-4610-88b0-f36a9ffc8c05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"
Jul 20 18:57:01.017822 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:57:01.017745    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-serving-cert: secret "maas-api-e2e-api-a-9eed89-serving-cert" not found
Jul 20 18:57:01.017822 ip-10-0-129-84 kubenswrapper[2569]: E0720 18:57:01.017810    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98498b18-74b8-4610-88b0-f36a9ffc8c05-maas-api-tls podName:98498b18-74b8-4610-88b0-f36a9ffc8c05 nodeName:}" failed. No retries permitted until 2026-07-20 18:57:01.517789016 +0000 UTC m=+2618.351761389 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/98498b18-74b8-4610-88b0-f36a9ffc8c05-maas-api-tls") pod "maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k" (UID: "98498b18-74b8-4610-88b0-f36a9ffc8c05") : secret "maas-api-e2e-api-a-9eed89-serving-cert" not found
Jul 20 18:57:01.018360 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:01.018337    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98498b18-74b8-4610-88b0-f36a9ffc8c05-openshift-service-ca\") pod \"maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k\" (UID: \"98498b18-74b8-4610-88b0-f36a9ffc8c05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"
Jul 20 18:57:01.026497 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:01.026471    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh2wk\" (UniqueName: \"kubernetes.io/projected/98498b18-74b8-4610-88b0-f36a9ffc8c05-kube-api-access-wh2wk\") pod \"maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k\" (UID: \"98498b18-74b8-4610-88b0-f36a9ffc8c05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"
Jul 20 18:57:01.306652 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:01.306618    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:01.523209 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:01.523175    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98498b18-74b8-4610-88b0-f36a9ffc8c05-maas-api-tls\") pod \"maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k\" (UID: \"98498b18-74b8-4610-88b0-f36a9ffc8c05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"
Jul 20 18:57:01.525652 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:01.525631    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98498b18-74b8-4610-88b0-f36a9ffc8c05-maas-api-tls\") pod \"maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k\" (UID: \"98498b18-74b8-4610-88b0-f36a9ffc8c05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"
Jul 20 18:57:01.623273 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:01.623143    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:01.724680 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:01.724626    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:01.811497 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:01.811467    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"
Jul 20 18:57:01.950519 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:01.950495    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"]
Jul 20 18:57:01.952658 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:57:01.952627    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98498b18_74b8_4610_88b0_f36a9ffc8c05.slice/crio-9e165c1eb2ca6454a1d86047d9b0c4adfeeb0f2f4e2ba7f9e01a58703feb930d WatchSource:0}: Error finding container 9e165c1eb2ca6454a1d86047d9b0c4adfeeb0f2f4e2ba7f9e01a58703feb930d: Status 404 returned error can't find the container with id 9e165c1eb2ca6454a1d86047d9b0c4adfeeb0f2f4e2ba7f9e01a58703feb930d
Jul 20 18:57:01.954032 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:01.954015    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 18:57:02.284872 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:02.284838    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k" event={"ID":"98498b18-74b8-4610-88b0-f36a9ffc8c05","Type":"ContainerStarted","Data":"9e165c1eb2ca6454a1d86047d9b0c4adfeeb0f2f4e2ba7f9e01a58703feb930d"}
Jul 20 18:57:03.292373 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:03.292336    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k" event={"ID":"98498b18-74b8-4610-88b0-f36a9ffc8c05","Type":"ContainerStarted","Data":"6c05b43cd734f8839e5527ed9f82d54bec8f563552b1a434c39f31b1057938b7"}
Jul 20 18:57:03.292795 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:03.292444    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"
Jul 20 18:57:03.310285 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:03.310233    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k" podStartSLOduration=2.495270592 podStartE2EDuration="3.310215917s" totalImagesPullingTime="814.945325ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:57:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:57:03.30923974 +0000 UTC m=+2620.143212131" watchObservedRunningTime="2026-07-20 18:57:03.310215917 +0000 UTC m=+2620.144188304"
Jul 20 18:57:05.278973 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:05.278940    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:05.936090 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:05.936043    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:06.071174 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:06.071140    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:06.395611 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:06.395570    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:09.223778 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:09.223751    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:09.303757 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:09.303730    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"
Jul 20 18:57:09.315154 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:09.315126    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:09.422519 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:09.422491    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:09.707546 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:09.707512    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:09.949941 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:09.949913    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:10.052023 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:10.051982    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:13.717866 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:13.717829    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:21.774672 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:21.774636    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9eed89-h7j22"]
Jul 20 18:57:21.779304 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:21.779283    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9eed89-h7j22"
Jul 20 18:57:21.781638 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:21.781614    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w8qkh\""
Jul 20 18:57:21.785298 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:21.785268    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9eed89-h7j22"]
Jul 20 18:57:21.925710 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:21.925677    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e77089c-40c7-4711-a375-c88218ce7579-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-9eed89-h7j22\" (UID: \"3e77089c-40c7-4711-a375-c88218ce7579\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9eed89-h7j22"
Jul 20 18:57:22.027117 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:22.026998    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e77089c-40c7-4711-a375-c88218ce7579-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-9eed89-h7j22\" (UID: \"3e77089c-40c7-4711-a375-c88218ce7579\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9eed89-h7j22"
Jul 20 18:57:22.027669 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:22.027647    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e77089c-40c7-4711-a375-c88218ce7579-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-9eed89-h7j22\" (UID: \"3e77089c-40c7-4711-a375-c88218ce7579\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9eed89-h7j22"
Jul 20 18:57:22.092326 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:22.092290    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9eed89-h7j22"
Jul 20 18:57:22.236825 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:22.236787    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9eed89-h7j22"]
Jul 20 18:57:22.238112 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:57:22.238081    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e77089c_40c7_4711_a375_c88218ce7579.slice/crio-7df745407fa11a9b6d7dd96b8ca37934751fa5242e96b84920e142429d58e5cc WatchSource:0}: Error finding container 7df745407fa11a9b6d7dd96b8ca37934751fa5242e96b84920e142429d58e5cc: Status 404 returned error can't find the container with id 7df745407fa11a9b6d7dd96b8ca37934751fa5242e96b84920e142429d58e5cc
Jul 20 18:57:22.401643 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:22.401602    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9eed89-h7j22" event={"ID":"3e77089c-40c7-4711-a375-c88218ce7579","Type":"ContainerStarted","Data":"7df745407fa11a9b6d7dd96b8ca37934751fa5242e96b84920e142429d58e5cc"}
Jul 20 18:57:22.490179 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:22.490148    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:22.591176 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:22.591142    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:23.408205 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:23.408171    2569 generic.go:356] "Generic (PLEG): container finished" podID="3e77089c-40c7-4711-a375-c88218ce7579" containerID="2140bb32123acdfeaba6e93955723a7a8c874adc87770aaacf9ed62735ae5c41" exitCode=0
Jul 20 18:57:23.408598 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:23.408258    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9eed89-h7j22" event={"ID":"3e77089c-40c7-4711-a375-c88218ce7579","Type":"ContainerDied","Data":"2140bb32123acdfeaba6e93955723a7a8c874adc87770aaacf9ed62735ae5c41"}
Jul 20 18:57:24.006003 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:24.005974    2569 scope.go:117] "RemoveContainer" containerID="87ffcf2474a549b2d1fcecdbceaa24ab90b1e40e6acf0d54c85073f66279c770"
Jul 20 18:57:24.547612 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:24.547584    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9eed89-h7j22"
Jul 20 18:57:24.652030 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:24.651995    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e77089c-40c7-4711-a375-c88218ce7579-maas-api-service-ca\") pod \"3e77089c-40c7-4711-a375-c88218ce7579\" (UID: \"3e77089c-40c7-4711-a375-c88218ce7579\") "
Jul 20 18:57:24.652339 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:24.652318    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e77089c-40c7-4711-a375-c88218ce7579-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3e77089c-40c7-4711-a375-c88218ce7579" (UID: "3e77089c-40c7-4711-a375-c88218ce7579"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:57:24.752919 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:24.752896    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e77089c-40c7-4711-a375-c88218ce7579-maas-api-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:57:25.420180 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:25.420146    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9eed89-h7j22"
Jul 20 18:57:25.420363 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:25.420177    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9eed89-h7j22" event={"ID":"3e77089c-40c7-4711-a375-c88218ce7579","Type":"ContainerDied","Data":"7df745407fa11a9b6d7dd96b8ca37934751fa5242e96b84920e142429d58e5cc"}
Jul 20 18:57:25.420363 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:25.420233    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7df745407fa11a9b6d7dd96b8ca37934751fa5242e96b84920e142429d58e5cc"
Jul 20 18:57:32.017758 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:32.017723    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:32.116686 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:32.116656    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:37.262854 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:37.262821    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"]
Jul 20 18:57:37.263428 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:37.263141    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k" podUID="98498b18-74b8-4610-88b0-f36a9ffc8c05" containerName="maas-api" containerID="cri-o://6c05b43cd734f8839e5527ed9f82d54bec8f563552b1a434c39f31b1057938b7" gracePeriod=30
Jul 20 18:57:37.291917 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:37.291889    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:37.479788 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:37.479756    2569 generic.go:356] "Generic (PLEG): container finished" podID="98498b18-74b8-4610-88b0-f36a9ffc8c05" containerID="6c05b43cd734f8839e5527ed9f82d54bec8f563552b1a434c39f31b1057938b7" exitCode=0
Jul 20 18:57:37.479960 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:37.479836    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k" event={"ID":"98498b18-74b8-4610-88b0-f36a9ffc8c05","Type":"ContainerDied","Data":"6c05b43cd734f8839e5527ed9f82d54bec8f563552b1a434c39f31b1057938b7"}
Jul 20 18:57:37.515071 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:37.514996    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"
Jul 20 18:57:37.565908 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:37.565874    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98498b18-74b8-4610-88b0-f36a9ffc8c05-openshift-service-ca\") pod \"98498b18-74b8-4610-88b0-f36a9ffc8c05\" (UID: \"98498b18-74b8-4610-88b0-f36a9ffc8c05\") "
Jul 20 18:57:37.566126 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:37.565927    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wh2wk\" (UniqueName: \"kubernetes.io/projected/98498b18-74b8-4610-88b0-f36a9ffc8c05-kube-api-access-wh2wk\") pod \"98498b18-74b8-4610-88b0-f36a9ffc8c05\" (UID: \"98498b18-74b8-4610-88b0-f36a9ffc8c05\") "
Jul 20 18:57:37.566126 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:37.566002    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98498b18-74b8-4610-88b0-f36a9ffc8c05-maas-api-tls\") pod \"98498b18-74b8-4610-88b0-f36a9ffc8c05\" (UID: \"98498b18-74b8-4610-88b0-f36a9ffc8c05\") "
Jul 20 18:57:37.566316 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:37.566294    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98498b18-74b8-4610-88b0-f36a9ffc8c05-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "98498b18-74b8-4610-88b0-f36a9ffc8c05" (UID: "98498b18-74b8-4610-88b0-f36a9ffc8c05"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:57:37.568133 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:37.568102    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98498b18-74b8-4610-88b0-f36a9ffc8c05-kube-api-access-wh2wk" (OuterVolumeSpecName: "kube-api-access-wh2wk") pod "98498b18-74b8-4610-88b0-f36a9ffc8c05" (UID: "98498b18-74b8-4610-88b0-f36a9ffc8c05"). InnerVolumeSpecName "kube-api-access-wh2wk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 18:57:37.568223 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:37.568201    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98498b18-74b8-4610-88b0-f36a9ffc8c05-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "98498b18-74b8-4610-88b0-f36a9ffc8c05" (UID: "98498b18-74b8-4610-88b0-f36a9ffc8c05"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 18:57:37.666680 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:37.666642    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wh2wk\" (UniqueName: \"kubernetes.io/projected/98498b18-74b8-4610-88b0-f36a9ffc8c05-kube-api-access-wh2wk\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:57:37.666680 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:37.666672    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98498b18-74b8-4610-88b0-f36a9ffc8c05-maas-api-tls\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:57:37.666680 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:37.666682    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98498b18-74b8-4610-88b0-f36a9ffc8c05-openshift-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:57:38.485968 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:38.485937    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"
Jul 20 18:57:38.486457 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:38.485985    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k" event={"ID":"98498b18-74b8-4610-88b0-f36a9ffc8c05","Type":"ContainerDied","Data":"9e165c1eb2ca6454a1d86047d9b0c4adfeeb0f2f4e2ba7f9e01a58703feb930d"}
Jul 20 18:57:38.486457 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:38.486074    2569 scope.go:117] "RemoveContainer" containerID="6c05b43cd734f8839e5527ed9f82d54bec8f563552b1a434c39f31b1057938b7"
Jul 20 18:57:38.506945 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:38.506918    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"]
Jul 20 18:57:38.511953 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:38.511929    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9eed89-6cf45b8b7f-ngg6k"]
Jul 20 18:57:39.759174 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:39.759140    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98498b18-74b8-4610-88b0-f36a9ffc8c05" path="/var/lib/kubelet/pods/98498b18-74b8-4610-88b0-f36a9ffc8c05/volumes"
Jul 20 18:57:41.867230 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:41.867197    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:41.868158 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:41.868129    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9eed89-h7j22"]
Jul 20 18:57:41.870105 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:41.870084    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9eed89-h7j22"]
Jul 20 18:57:42.259268 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:42.259234    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:43.758731 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:43.758699    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e77089c-40c7-4711-a375-c88218ce7579" path="/var/lib/kubelet/pods/3e77089c-40c7-4711-a375-c88218ce7579/volumes"
Jul 20 18:57:47.973097 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:47.973040    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:48.085291 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.085255    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:48.280066 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.280024    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:48.500368 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.500335    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:48.615886 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.615805    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp"]
Jul 20 18:57:48.616262 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.616248    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e77089c-40c7-4711-a375-c88218ce7579" containerName="revoke-keys"
Jul 20 18:57:48.616310 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.616264    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e77089c-40c7-4711-a375-c88218ce7579" containerName="revoke-keys"
Jul 20 18:57:48.616310 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.616288    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98498b18-74b8-4610-88b0-f36a9ffc8c05" containerName="maas-api"
Jul 20 18:57:48.616310 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.616294    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="98498b18-74b8-4610-88b0-f36a9ffc8c05" containerName="maas-api"
Jul 20 18:57:48.616403 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.616357    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="98498b18-74b8-4610-88b0-f36a9ffc8c05" containerName="maas-api"
Jul 20 18:57:48.616403 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.616366    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e77089c-40c7-4711-a375-c88218ce7579" containerName="revoke-keys"
Jul 20 18:57:48.620889 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.620868    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp"
Jul 20 18:57:48.623074 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.623033    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w8qkh\""
Jul 20 18:57:48.627341 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.627317    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp"]
Jul 20 18:57:48.668024 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.667987    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0de2e9-3ce4-47b6-86e9-8537e883da89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-15b416-xfhgp\" (UID: \"4b0de2e9-3ce4-47b6-86e9-8537e883da89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp"
Jul 20 18:57:48.769645 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.769598    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0de2e9-3ce4-47b6-86e9-8537e883da89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-15b416-xfhgp\" (UID: \"4b0de2e9-3ce4-47b6-86e9-8537e883da89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp"
Jul 20 18:57:48.770292 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.770273    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0de2e9-3ce4-47b6-86e9-8537e883da89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-15b416-xfhgp\" (UID: \"4b0de2e9-3ce4-47b6-86e9-8537e883da89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp"
Jul 20 18:57:48.933814 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:48.933723    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp"
Jul 20 18:57:49.069391 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:49.069362    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp"]
Jul 20 18:57:49.072919 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:57:49.072891    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b0de2e9_3ce4_47b6_86e9_8537e883da89.slice/crio-960be99beba76cb08358ec088d9524e02f81df1b8d613f1cb3d4616d3f95c7a3 WatchSource:0}: Error finding container 960be99beba76cb08358ec088d9524e02f81df1b8d613f1cb3d4616d3f95c7a3: Status 404 returned error can't find the container with id 960be99beba76cb08358ec088d9524e02f81df1b8d613f1cb3d4616d3f95c7a3
Jul 20 18:57:49.542866 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:49.542829    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp" event={"ID":"4b0de2e9-3ce4-47b6-86e9-8537e883da89","Type":"ContainerStarted","Data":"8c307d04dab8e7d9d87eda4ccfc344b3ad59e660c1094574d05e93caf56e5b85"}
Jul 20 18:57:49.543039 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:49.542897    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp" event={"ID":"4b0de2e9-3ce4-47b6-86e9-8537e883da89","Type":"ContainerStarted","Data":"960be99beba76cb08358ec088d9524e02f81df1b8d613f1cb3d4616d3f95c7a3"}
Jul 20 18:57:49.558097 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:49.558028    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp" podStartSLOduration=1.5580112069999998 podStartE2EDuration="1.558011207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:57:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:57:49.557320616 +0000 UTC m=+2666.391293008" watchObservedRunningTime="2026-07-20 18:57:49.558011207 +0000 UTC m=+2666.391983594"
Jul 20 18:57:50.550105 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:50.550066    2569 generic.go:356] "Generic (PLEG): container finished" podID="4b0de2e9-3ce4-47b6-86e9-8537e883da89" containerID="8c307d04dab8e7d9d87eda4ccfc344b3ad59e660c1094574d05e93caf56e5b85" exitCode=0
Jul 20 18:57:50.550487 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:50.550147    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp" event={"ID":"4b0de2e9-3ce4-47b6-86e9-8537e883da89","Type":"ContainerDied","Data":"8c307d04dab8e7d9d87eda4ccfc344b3ad59e660c1094574d05e93caf56e5b85"}
Jul 20 18:57:51.688342 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:51.688318    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp"
Jul 20 18:57:51.796855 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:51.796819    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0de2e9-3ce4-47b6-86e9-8537e883da89-maas-api-service-ca\") pod \"4b0de2e9-3ce4-47b6-86e9-8537e883da89\" (UID: \"4b0de2e9-3ce4-47b6-86e9-8537e883da89\") "
Jul 20 18:57:51.797262 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:51.797235    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b0de2e9-3ce4-47b6-86e9-8537e883da89-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4b0de2e9-3ce4-47b6-86e9-8537e883da89" (UID: "4b0de2e9-3ce4-47b6-86e9-8537e883da89"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 18:57:51.898001 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:51.897920    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0de2e9-3ce4-47b6-86e9-8537e883da89-maas-api-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 18:57:52.561272 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:52.561238    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp" event={"ID":"4b0de2e9-3ce4-47b6-86e9-8537e883da89","Type":"ContainerDied","Data":"960be99beba76cb08358ec088d9524e02f81df1b8d613f1cb3d4616d3f95c7a3"}
Jul 20 18:57:52.561272 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:52.561269    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp"
Jul 20 18:57:52.561488 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:52.561300    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="960be99beba76cb08358ec088d9524e02f81df1b8d613f1cb3d4616d3f95c7a3"
Jul 20 18:57:54.215942 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:54.215912    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:54.314564 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:54.314535    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:59.362689 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:59.362655    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:57:59.462063 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:57:59.462031    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:04.643944 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:04.643901    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:08.705165 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:08.705129    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp"]
Jul 20 18:58:08.707855 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:08.707833    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-15b416-xfhgp"]
Jul 20 18:58:08.714179 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:08.714158    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:09.128602 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:09.128542    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:09.759220 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:09.759189    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b0de2e9-3ce4-47b6-86e9-8537e883da89" path="/var/lib/kubelet/pods/4b0de2e9-3ce4-47b6-86e9-8537e883da89/volumes"
Jul 20 18:58:10.879615 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:10.879581    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:14.782908 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:14.782872    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:14.883219 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:14.883186    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:15.045476 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:15.045310    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:15.275566 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:15.275534    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:15.605329 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:15.605298    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:15.910755 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:15.910670    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:16.074143 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:16.074085    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:19.169342 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.169297    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:19.195686 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.195654    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"]
Jul 20 18:58:19.196221 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.196202    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b0de2e9-3ce4-47b6-86e9-8537e883da89" containerName="revoke-keys"
Jul 20 18:58:19.196298 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.196224    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b0de2e9-3ce4-47b6-86e9-8537e883da89" containerName="revoke-keys"
Jul 20 18:58:19.196342 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.196313    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b0de2e9-3ce4-47b6-86e9-8537e883da89" containerName="revoke-keys"
Jul 20 18:58:19.199581 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.199564    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"
Jul 20 18:58:19.202241 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.202222    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-b7bb94-serving-cert\""
Jul 20 18:58:19.202691 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.202670    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vvgb2\""
Jul 20 18:58:19.212382 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.212360    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"]
Jul 20 18:58:19.267828 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.267788    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:19.363174 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.363136    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4573384e-dae5-4660-80e4-3a4959d5a09f-maas-api-tls\") pod \"maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx\" (UID: \"4573384e-dae5-4660-80e4-3a4959d5a09f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"
Jul 20 18:58:19.363319 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.363196    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4573384e-dae5-4660-80e4-3a4959d5a09f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx\" (UID: \"4573384e-dae5-4660-80e4-3a4959d5a09f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"
Jul 20 18:58:19.363319 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.363224    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-599c4\" (UniqueName: \"kubernetes.io/projected/4573384e-dae5-4660-80e4-3a4959d5a09f-kube-api-access-599c4\") pod \"maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx\" (UID: \"4573384e-dae5-4660-80e4-3a4959d5a09f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"
Jul 20 18:58:19.370599 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.370578    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:19.464026 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.463928    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4573384e-dae5-4660-80e4-3a4959d5a09f-maas-api-tls\") pod \"maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx\" (UID: \"4573384e-dae5-4660-80e4-3a4959d5a09f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"
Jul 20 18:58:19.464026 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.464015    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4573384e-dae5-4660-80e4-3a4959d5a09f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx\" (UID: \"4573384e-dae5-4660-80e4-3a4959d5a09f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"
Jul 20 18:58:19.464267 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.464049    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-599c4\" (UniqueName: \"kubernetes.io/projected/4573384e-dae5-4660-80e4-3a4959d5a09f-kube-api-access-599c4\") pod \"maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx\" (UID: \"4573384e-dae5-4660-80e4-3a4959d5a09f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"
Jul 20 18:58:19.464674 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.464651    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4573384e-dae5-4660-80e4-3a4959d5a09f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx\" (UID: \"4573384e-dae5-4660-80e4-3a4959d5a09f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"
Jul 20 18:58:19.466409 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.466389    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4573384e-dae5-4660-80e4-3a4959d5a09f-maas-api-tls\") pod \"maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx\" (UID: \"4573384e-dae5-4660-80e4-3a4959d5a09f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"
Jul 20 18:58:19.472973 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.472940    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-599c4\" (UniqueName: \"kubernetes.io/projected/4573384e-dae5-4660-80e4-3a4959d5a09f-kube-api-access-599c4\") pod \"maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx\" (UID: \"4573384e-dae5-4660-80e4-3a4959d5a09f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"
Jul 20 18:58:19.509975 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.509948    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"
Jul 20 18:58:19.849355 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.849320    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"]
Jul 20 18:58:19.851577 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:58:19.851551    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4573384e_dae5_4660_80e4_3a4959d5a09f.slice/crio-8c643ded9d694ed77ae986ebb38472ff56f7dcad8bcfa38ebff079d622631ed2 WatchSource:0}: Error finding container 8c643ded9d694ed77ae986ebb38472ff56f7dcad8bcfa38ebff079d622631ed2: Status 404 returned error can't find the container with id 8c643ded9d694ed77ae986ebb38472ff56f7dcad8bcfa38ebff079d622631ed2
Jul 20 18:58:19.923866 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:19.923807    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:20.024731 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:20.024698    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:20.714744 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:20.714657    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx" event={"ID":"4573384e-dae5-4660-80e4-3a4959d5a09f","Type":"ContainerStarted","Data":"2a1615eee838fdf5b8f1cf870f612b1154f87fbc7e78547c524a406326609e32"}
Jul 20 18:58:20.714744 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:20.714714    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"
Jul 20 18:58:20.714744 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:20.714739    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx" event={"ID":"4573384e-dae5-4660-80e4-3a4959d5a09f","Type":"ContainerStarted","Data":"8c643ded9d694ed77ae986ebb38472ff56f7dcad8bcfa38ebff079d622631ed2"}
Jul 20 18:58:20.732691 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:20.732644    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx" podStartSLOduration=1.188316128 podStartE2EDuration="1.732629467s" totalImagesPullingTime="544.313339ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:58:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:58:20.730966528 +0000 UTC m=+2697.564938915" watchObservedRunningTime="2026-07-20 18:58:20.732629467 +0000 UTC m=+2697.566601854"
Jul 20 18:58:23.593433 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:23.593399    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:23.940109 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:23.940019    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:24.037601 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:24.037568    2569 scope.go:117] "RemoveContainer" containerID="860647bdaecdc5ae29b51ee7935fbd76255f1a360403af7cc1f095dc9f14467d"
Jul 20 18:58:24.266677 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:24.266641    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:24.421718 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:24.421686    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:26.721890 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:26.721854    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:26.731078 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:26.731042    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"
Jul 20 18:58:26.881946 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:26.881888    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:27.050269 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:27.050239    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:27.130106 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:27.130040    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:27.491690 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:27.491608    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:27.590690 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:27.590655    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:30.372724 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:30.372618    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:58:30.390733 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:30.385408    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 18:58:31.261214 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:31.261185    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:40.939561 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:40.939526    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:41.039009 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:41.038974    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:48.253042 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:48.253004    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:48.354192 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:48.354161    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:49.393637 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.393596    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"]
Jul 20 18:58:49.397285 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.397266    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.400698 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.400675    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b7bb94\"/\"default-dockercfg-27j9g\""
Jul 20 18:58:49.400800 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.400698    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b7bb94\"/\"openshift-service-ca.crt\""
Jul 20 18:58:49.400800 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.400711    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b7bb94\"/\"auth-test-model-b7bb94-kserve-self-signed-certs\""
Jul 20 18:58:49.400800 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.400707    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b7bb94\"/\"kube-root-ca.crt\""
Jul 20 18:58:49.406604 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.406583    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"]
Jul 20 18:58:49.474354 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.474299    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 18:58:49.542111 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.542078    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-dshm\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.542278 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.542180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-tmp-dir\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.542278 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.542207    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-model-cache\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.542278 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.542228    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-home\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.542278 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.542255    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0899eebd-4c99-41b3-9899-19a889e39aca-tls-certs\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.542432 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.542349    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk8bf\" (UniqueName: \"kubernetes.io/projected/0899eebd-4c99-41b3-9899-19a889e39aca-kube-api-access-bk8bf\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.643776 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.643693    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-tmp-dir\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.643776 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.643726    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-model-cache\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.643776 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.643745    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-home\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.643776 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.643768    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0899eebd-4c99-41b3-9899-19a889e39aca-tls-certs\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.644111 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.643817    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk8bf\" (UniqueName: \"kubernetes.io/projected/0899eebd-4c99-41b3-9899-19a889e39aca-kube-api-access-bk8bf\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.644111 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.643837    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-dshm\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.644232 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.644148    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-tmp-dir\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.644232 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.644205    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-model-cache\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.644304 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.644239    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-home\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.646114 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.646085    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-dshm\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.646242 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.646225    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0899eebd-4c99-41b3-9899-19a889e39aca-tls-certs\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.651765 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.651746    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk8bf\" (UniqueName: \"kubernetes.io/projected/0899eebd-4c99-41b3-9899-19a889e39aca-kube-api-access-bk8bf\") pod \"auth-test-model-b7bb94-kserve-58475b6755-88fhk\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") " pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.708161 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.708118    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:49.841447 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.841423    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"]
Jul 20 18:58:49.843628 ip-10-0-129-84 kubenswrapper[2569]: W0720 18:58:49.843597    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0899eebd_4c99_41b3_9899_19a889e39aca.slice/crio-24a547d55fe5bedc867bf9f0d8a2fba46e855bd22d297eed4eba4e41c9542a34 WatchSource:0}: Error finding container 24a547d55fe5bedc867bf9f0d8a2fba46e855bd22d297eed4eba4e41c9542a34: Status 404 returned error can't find the container with id 24a547d55fe5bedc867bf9f0d8a2fba46e855bd22d297eed4eba4e41c9542a34
Jul 20 18:58:49.869636 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:49.869609    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk" event={"ID":"0899eebd-4c99-41b3-9899-19a889e39aca","Type":"ContainerStarted","Data":"24a547d55fe5bedc867bf9f0d8a2fba46e855bd22d297eed4eba4e41c9542a34"}
Jul 20 18:58:50.875971 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:50.875934    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk" event={"ID":"0899eebd-4c99-41b3-9899-19a889e39aca","Type":"ContainerStarted","Data":"741d61e6cf140256ba6a7a1420ad8daeb49268a4530da13a7c6a0f71ad57e8d5"}
Jul 20 18:58:50.892580 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:50.892532    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk" podStartSLOduration=1.892516442 podStartE2EDuration="1.892516442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 18:58:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 18:58:50.891240181 +0000 UTC m=+2727.725212571" watchObservedRunningTime="2026-07-20 18:58:50.892516442 +0000 UTC m=+2727.726488829"
Jul 20 18:58:59.708970 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:59.708934    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:59.708970 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:59.708979    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:59.721521 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:59.721497    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:58:59.932916 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:58:59.932890    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 18:59:13.109251 ip-10-0-129-84 kubenswrapper[2569]: I0720 18:59:13.109219    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:00:00.136504 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:00.136467    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b7bb94-29742900-gd7gf"]
Jul 20 19:00:00.140476 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:00.140453    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2f866-29742900-9cj45"]
Jul 20 19:00:00.140621 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:00.140568    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b7bb94-29742900-gd7gf"
Jul 20 19:00:00.143191 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:00.143166    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w8qkh\""
Jul 20 19:00:00.143669 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:00.143653    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2f866-29742900-9cj45"
Jul 20 19:00:00.148874 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:00.148432    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b7bb94-29742900-gd7gf"]
Jul 20 19:00:00.151752 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:00.151724    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2f866-29742900-9cj45"]
Jul 20 19:00:00.155180 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:00.155155    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b7bb94-29742900-gd7gf"
Jul 20 19:00:00.163707 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:00.163680    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2f866-29742900-9cj45"
Jul 20 19:00:00.314010 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:00.313887    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b7bb94-29742900-gd7gf"]
Jul 20 19:00:00.326014 ip-10-0-129-84 kubenswrapper[2569]: W0720 19:00:00.325977    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38a9d02b_de71_4814_a146_d8af0319fef9.slice/crio-4e6777d21d9796e8c9b4c94888b45badfcc1f232be206466ab6ce7201043076a WatchSource:0}: Error finding container 4e6777d21d9796e8c9b4c94888b45badfcc1f232be206466ab6ce7201043076a: Status 404 returned error can't find the container with id 4e6777d21d9796e8c9b4c94888b45badfcc1f232be206466ab6ce7201043076a
Jul 20 19:00:00.335366 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:00.335339    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2f866-29742900-9cj45"]
Jul 20 19:00:00.336955 ip-10-0-129-84 kubenswrapper[2569]: W0720 19:00:00.336928    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e771d13_5037_4d84_8200_6d2bfdd33e38.slice/crio-a5c92ce7b0c06b08b538e0a9c1fb5e8edb5ad90d1e8024254877947e50879f39 WatchSource:0}: Error finding container a5c92ce7b0c06b08b538e0a9c1fb5e8edb5ad90d1e8024254877947e50879f39: Status 404 returned error can't find the container with id a5c92ce7b0c06b08b538e0a9c1fb5e8edb5ad90d1e8024254877947e50879f39
Jul 20 19:00:01.238525 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:01.238465    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b7bb94-29742900-gd7gf" event={"ID":"38a9d02b-de71-4814-a146-d8af0319fef9","Type":"ContainerStarted","Data":"d064aec693dac640e3507bdaca4d79b0744dda75c5bd55d4a48e137bf89633a2"}
Jul 20 19:00:01.239088 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:01.238568    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b7bb94-29742900-gd7gf" event={"ID":"38a9d02b-de71-4814-a146-d8af0319fef9","Type":"ContainerStarted","Data":"4e6777d21d9796e8c9b4c94888b45badfcc1f232be206466ab6ce7201043076a"}
Jul 20 19:00:01.240581 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:01.240541    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2f866-29742900-9cj45" event={"ID":"4e771d13-5037-4d84-8200-6d2bfdd33e38","Type":"ContainerStarted","Data":"a8e2844cc3fa3b5e9d9eb8b60bfad86a6db4535b3c87a18353e11df77cc51443"}
Jul 20 19:00:01.240703 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:01.240609    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2f866-29742900-9cj45" event={"ID":"4e771d13-5037-4d84-8200-6d2bfdd33e38","Type":"ContainerStarted","Data":"a5c92ce7b0c06b08b538e0a9c1fb5e8edb5ad90d1e8024254877947e50879f39"}
Jul 20 19:00:01.255045 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:01.254995    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b7bb94-29742900-gd7gf" podStartSLOduration=1.254978511 podStartE2EDuration="1.254978511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:00:01.253524666 +0000 UTC m=+2798.087497054" watchObservedRunningTime="2026-07-20 19:00:01.254978511 +0000 UTC m=+2798.088950899"
Jul 20 19:00:01.269707 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:01.269657    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2f866-29742900-9cj45" podStartSLOduration=1.2696391089999999 podStartE2EDuration="1.269639109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:00:01.26914879 +0000 UTC m=+2798.103121180" watchObservedRunningTime="2026-07-20 19:00:01.269639109 +0000 UTC m=+2798.103611546"
Jul 20 19:00:02.247791 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:02.247752    2569 generic.go:356] "Generic (PLEG): container finished" podID="38a9d02b-de71-4814-a146-d8af0319fef9" containerID="d064aec693dac640e3507bdaca4d79b0744dda75c5bd55d4a48e137bf89633a2" exitCode=0
Jul 20 19:00:02.248267 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:02.247838    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b7bb94-29742900-gd7gf" event={"ID":"38a9d02b-de71-4814-a146-d8af0319fef9","Type":"ContainerDied","Data":"d064aec693dac640e3507bdaca4d79b0744dda75c5bd55d4a48e137bf89633a2"}
Jul 20 19:00:02.249487 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:02.249460    2569 generic.go:356] "Generic (PLEG): container finished" podID="4e771d13-5037-4d84-8200-6d2bfdd33e38" containerID="a8e2844cc3fa3b5e9d9eb8b60bfad86a6db4535b3c87a18353e11df77cc51443" exitCode=0
Jul 20 19:00:02.249585 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:02.249547    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2f866-29742900-9cj45" event={"ID":"4e771d13-5037-4d84-8200-6d2bfdd33e38","Type":"ContainerDied","Data":"a8e2844cc3fa3b5e9d9eb8b60bfad86a6db4535b3c87a18353e11df77cc51443"}
Jul 20 19:00:03.405466 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:03.405445    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2f866-29742900-9cj45"
Jul 20 19:00:03.408317 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:03.408295    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b7bb94-29742900-gd7gf"
Jul 20 19:00:04.261642 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:04.261611    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b7bb94-29742900-gd7gf"
Jul 20 19:00:04.261854 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:04.261611    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b7bb94-29742900-gd7gf" event={"ID":"38a9d02b-de71-4814-a146-d8af0319fef9","Type":"ContainerDied","Data":"4e6777d21d9796e8c9b4c94888b45badfcc1f232be206466ab6ce7201043076a"}
Jul 20 19:00:04.261854 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:04.261730    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e6777d21d9796e8c9b4c94888b45badfcc1f232be206466ab6ce7201043076a"
Jul 20 19:00:04.262981 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:04.262960    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2f866-29742900-9cj45" event={"ID":"4e771d13-5037-4d84-8200-6d2bfdd33e38","Type":"ContainerDied","Data":"a5c92ce7b0c06b08b538e0a9c1fb5e8edb5ad90d1e8024254877947e50879f39"}
Jul 20 19:00:04.263093 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:04.262992    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2f866-29742900-9cj45"
Jul 20 19:00:04.263093 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:00:04.262996    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5c92ce7b0c06b08b538e0a9c1fb5e8edb5ad90d1e8024254877947e50879f39"
Jul 20 19:01:15.280166 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:15.280095    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"]
Jul 20 19:01:15.358306 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:15.358270    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:01:18.347479 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:18.347435    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:01:18.855123 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:18.855088    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:01:24.085157 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.085127    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"]
Jul 20 19:01:24.085638 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.085621    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e771d13-5037-4d84-8200-6d2bfdd33e38" containerName="cleanup"
Jul 20 19:01:24.085748 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.085640    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e771d13-5037-4d84-8200-6d2bfdd33e38" containerName="cleanup"
Jul 20 19:01:24.085748 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.085652    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38a9d02b-de71-4814-a146-d8af0319fef9" containerName="cleanup"
Jul 20 19:01:24.085748 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.085658    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="38a9d02b-de71-4814-a146-d8af0319fef9" containerName="cleanup"
Jul 20 19:01:24.085748 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.085726    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e771d13-5037-4d84-8200-6d2bfdd33e38" containerName="cleanup"
Jul 20 19:01:24.085944 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.085748    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="38a9d02b-de71-4814-a146-d8af0319fef9" containerName="cleanup"
Jul 20 19:01:24.088178 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.088150    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.090661 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.090641    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b7bb94\"/\"sub-test-model-b7bb94-kserve-self-signed-certs\""
Jul 20 19:01:24.100315 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.100293    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"]
Jul 20 19:01:24.123794 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.123766    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:01:24.204247 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.204218    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-dshm\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.204415 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.204267    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-tmp-dir\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.204415 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.204332    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-model-cache\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.204415 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.204358    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8prn\" (UniqueName: \"kubernetes.io/projected/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-kube-api-access-k8prn\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.204415 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.204380    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-home\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.204564 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.204432    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-tls-certs\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.305683 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.305646    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8prn\" (UniqueName: \"kubernetes.io/projected/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-kube-api-access-k8prn\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.305683 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.305687    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-home\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.305898 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.305753    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-tls-certs\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.305898 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.305795    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-dshm\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.305898 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.305835    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-tmp-dir\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.305898 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.305873    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-model-cache\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.306181 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.306161    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-home\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.306248 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.306196    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-model-cache\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.306324 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.306304    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-tmp-dir\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.308075 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.308041    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-dshm\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.308409 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.308393    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-tls-certs\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.314150 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.314128    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8prn\" (UniqueName: \"kubernetes.io/projected/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-kube-api-access-k8prn\") pod \"sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") " pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.398910 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.398836    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:24.540250 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.540222    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"]
Jul 20 19:01:24.544991 ip-10-0-129-84 kubenswrapper[2569]: W0720 19:01:24.542265    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8b9483a_9f70_4c4b_be7b_711fd6e8dee1.slice/crio-ba4ec07dd776aa77f38e32aead8f07ed3e65c992a87ceb1d14923a45a45ca01b WatchSource:0}: Error finding container ba4ec07dd776aa77f38e32aead8f07ed3e65c992a87ceb1d14923a45a45ca01b: Status 404 returned error can't find the container with id ba4ec07dd776aa77f38e32aead8f07ed3e65c992a87ceb1d14923a45a45ca01b
Jul 20 19:01:24.680987 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.680901    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd" event={"ID":"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1","Type":"ContainerStarted","Data":"48c175384c0f309b9c4cf3d33ea46b4c4e50718f1bb1b5c79252971d8ec5f812"}
Jul 20 19:01:24.680987 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.680943    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd" event={"ID":"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1","Type":"ContainerStarted","Data":"ba4ec07dd776aa77f38e32aead8f07ed3e65c992a87ceb1d14923a45a45ca01b"}
Jul 20 19:01:24.697753 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:24.697707    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd" podStartSLOduration=0.697691172 podStartE2EDuration="697.691172ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:01:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:01:24.696445032 +0000 UTC m=+2881.530417421" watchObservedRunningTime="2026-07-20 19:01:24.697691172 +0000 UTC m=+2881.531663559"
Jul 20 19:01:25.254206 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:25.254168    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:01:30.300608 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.300543    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk" podUID="0899eebd-4c99-41b3-9899-19a889e39aca" containerName="main" containerID="cri-o://741d61e6cf140256ba6a7a1420ad8daeb49268a4530da13a7c6a0f71ad57e8d5" gracePeriod=45
Jul 20 19:01:30.558627 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.558571    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 19:01:30.670373 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.670336    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-home\") pod \"0899eebd-4c99-41b3-9899-19a889e39aca\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") "
Jul 20 19:01:30.670373 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.670379    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bk8bf\" (UniqueName: \"kubernetes.io/projected/0899eebd-4c99-41b3-9899-19a889e39aca-kube-api-access-bk8bf\") pod \"0899eebd-4c99-41b3-9899-19a889e39aca\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") "
Jul 20 19:01:30.670623 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.670432    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-tmp-dir\") pod \"0899eebd-4c99-41b3-9899-19a889e39aca\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") "
Jul 20 19:01:30.670623 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.670480    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-dshm\") pod \"0899eebd-4c99-41b3-9899-19a889e39aca\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") "
Jul 20 19:01:30.670623 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.670558    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-model-cache\") pod \"0899eebd-4c99-41b3-9899-19a889e39aca\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") "
Jul 20 19:01:30.670623 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.670598    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0899eebd-4c99-41b3-9899-19a889e39aca-tls-certs\") pod \"0899eebd-4c99-41b3-9899-19a889e39aca\" (UID: \"0899eebd-4c99-41b3-9899-19a889e39aca\") "
Jul 20 19:01:30.670623 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.670598    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-home" (OuterVolumeSpecName: "home") pod "0899eebd-4c99-41b3-9899-19a889e39aca" (UID: "0899eebd-4c99-41b3-9899-19a889e39aca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 19:01:30.670851 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.670716    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0899eebd-4c99-41b3-9899-19a889e39aca" (UID: "0899eebd-4c99-41b3-9899-19a889e39aca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 19:01:30.670916 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.670848    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-model-cache" (OuterVolumeSpecName: "model-cache") pod "0899eebd-4c99-41b3-9899-19a889e39aca" (UID: "0899eebd-4c99-41b3-9899-19a889e39aca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 19:01:30.670994 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.670977    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-home\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:01:30.671069 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.670996    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-tmp-dir\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:01:30.671069 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.671005    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-model-cache\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:01:30.672623 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.672602    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-dshm" (OuterVolumeSpecName: "dshm") pod "0899eebd-4c99-41b3-9899-19a889e39aca" (UID: "0899eebd-4c99-41b3-9899-19a889e39aca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 19:01:30.672730 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.672660    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0899eebd-4c99-41b3-9899-19a889e39aca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0899eebd-4c99-41b3-9899-19a889e39aca" (UID: "0899eebd-4c99-41b3-9899-19a889e39aca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 19:01:30.672730 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.672695    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0899eebd-4c99-41b3-9899-19a889e39aca-kube-api-access-bk8bf" (OuterVolumeSpecName: "kube-api-access-bk8bf") pod "0899eebd-4c99-41b3-9899-19a889e39aca" (UID: "0899eebd-4c99-41b3-9899-19a889e39aca"). InnerVolumeSpecName "kube-api-access-bk8bf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 19:01:30.713279 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.713240    2569 generic.go:356] "Generic (PLEG): container finished" podID="0899eebd-4c99-41b3-9899-19a889e39aca" containerID="741d61e6cf140256ba6a7a1420ad8daeb49268a4530da13a7c6a0f71ad57e8d5" exitCode=0
Jul 20 19:01:30.713279 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.713284    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"
Jul 20 19:01:30.713527 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.713323    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk" event={"ID":"0899eebd-4c99-41b3-9899-19a889e39aca","Type":"ContainerDied","Data":"741d61e6cf140256ba6a7a1420ad8daeb49268a4530da13a7c6a0f71ad57e8d5"}
Jul 20 19:01:30.713527 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.713387    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk" event={"ID":"0899eebd-4c99-41b3-9899-19a889e39aca","Type":"ContainerDied","Data":"24a547d55fe5bedc867bf9f0d8a2fba46e855bd22d297eed4eba4e41c9542a34"}
Jul 20 19:01:30.713527 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.713409    2569 scope.go:117] "RemoveContainer" containerID="741d61e6cf140256ba6a7a1420ad8daeb49268a4530da13a7c6a0f71ad57e8d5"
Jul 20 19:01:30.722567 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.722552    2569 scope.go:117] "RemoveContainer" containerID="741d61e6cf140256ba6a7a1420ad8daeb49268a4530da13a7c6a0f71ad57e8d5"
Jul 20 19:01:30.722842 ip-10-0-129-84 kubenswrapper[2569]: E0720 19:01:30.722823    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"741d61e6cf140256ba6a7a1420ad8daeb49268a4530da13a7c6a0f71ad57e8d5\": container with ID starting with 741d61e6cf140256ba6a7a1420ad8daeb49268a4530da13a7c6a0f71ad57e8d5 not found: ID does not exist" containerID="741d61e6cf140256ba6a7a1420ad8daeb49268a4530da13a7c6a0f71ad57e8d5"
Jul 20 19:01:30.722887 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.722852    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"741d61e6cf140256ba6a7a1420ad8daeb49268a4530da13a7c6a0f71ad57e8d5"} err="failed to get container status \"741d61e6cf140256ba6a7a1420ad8daeb49268a4530da13a7c6a0f71ad57e8d5\": rpc error: code = NotFound desc = could not find container \"741d61e6cf140256ba6a7a1420ad8daeb49268a4530da13a7c6a0f71ad57e8d5\": container with ID starting with 741d61e6cf140256ba6a7a1420ad8daeb49268a4530da13a7c6a0f71ad57e8d5 not found: ID does not exist"
Jul 20 19:01:30.734048 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.734026    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"]
Jul 20 19:01:30.737545 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.737522    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b7bb94/auth-test-model-b7bb94-kserve-58475b6755-88fhk"]
Jul 20 19:01:30.772117 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.772084    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0899eebd-4c99-41b3-9899-19a889e39aca-tls-certs\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:01:30.772117 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.772111    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bk8bf\" (UniqueName: \"kubernetes.io/projected/0899eebd-4c99-41b3-9899-19a889e39aca-kube-api-access-bk8bf\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:01:30.772117 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:30.772123    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0899eebd-4c99-41b3-9899-19a889e39aca-dshm\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:01:31.759380 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:31.759349    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0899eebd-4c99-41b3-9899-19a889e39aca" path="/var/lib/kubelet/pods/0899eebd-4c99-41b3-9899-19a889e39aca/volumes"
Jul 20 19:01:33.075849 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:33.075814    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:01:33.176261 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:33.176227    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:01:34.399126 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:34.399087    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:34.399528 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:34.399136    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:34.411362 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:34.411337    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:34.746361 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:34.746332    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:01:55.215071 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:55.215018    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:01:57.112886 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:01:57.112853    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:02:33.558550 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:33.558517    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"]
Jul 20 19:02:33.584337 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:33.584286    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:02:34.880220 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:34.880187    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:02:41.518728 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:41.518688    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:02:48.580632 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:48.580591    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd" podUID="a8b9483a-9f70-4c4b-be7b-711fd6e8dee1" containerName="main" containerID="cri-o://48c175384c0f309b9c4cf3d33ea46b4c4e50718f1bb1b5c79252971d8ec5f812" gracePeriod=45
Jul 20 19:02:48.838849 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:48.838796    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:02:48.905859 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:48.905822    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-dshm\") pod \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") "
Jul 20 19:02:48.906082 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:48.905890    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8prn\" (UniqueName: \"kubernetes.io/projected/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-kube-api-access-k8prn\") pod \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") "
Jul 20 19:02:48.906082 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:48.905919    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-tls-certs\") pod \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") "
Jul 20 19:02:48.906082 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:48.905973    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-home\") pod \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") "
Jul 20 19:02:48.906082 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:48.906000    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-model-cache\") pod \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") "
Jul 20 19:02:48.906082 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:48.906037    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-tmp-dir\") pod \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\" (UID: \"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1\") "
Jul 20 19:02:48.906371 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:48.906294    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-home" (OuterVolumeSpecName: "home") pod "a8b9483a-9f70-4c4b-be7b-711fd6e8dee1" (UID: "a8b9483a-9f70-4c4b-be7b-711fd6e8dee1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 19:02:48.906371 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:48.906308    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-model-cache" (OuterVolumeSpecName: "model-cache") pod "a8b9483a-9f70-4c4b-be7b-711fd6e8dee1" (UID: "a8b9483a-9f70-4c4b-be7b-711fd6e8dee1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 19:02:48.906371 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:48.906308    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a8b9483a-9f70-4c4b-be7b-711fd6e8dee1" (UID: "a8b9483a-9f70-4c4b-be7b-711fd6e8dee1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 19:02:48.908091 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:48.908072    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-dshm" (OuterVolumeSpecName: "dshm") pod "a8b9483a-9f70-4c4b-be7b-711fd6e8dee1" (UID: "a8b9483a-9f70-4c4b-be7b-711fd6e8dee1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 19:02:48.908164 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:48.908117    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-kube-api-access-k8prn" (OuterVolumeSpecName: "kube-api-access-k8prn") pod "a8b9483a-9f70-4c4b-be7b-711fd6e8dee1" (UID: "a8b9483a-9f70-4c4b-be7b-711fd6e8dee1"). InnerVolumeSpecName "kube-api-access-k8prn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 19:02:48.908211 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:48.908197    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a8b9483a-9f70-4c4b-be7b-711fd6e8dee1" (UID: "a8b9483a-9f70-4c4b-be7b-711fd6e8dee1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 19:02:49.007420 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:49.007378    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-tls-certs\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:02:49.007420 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:49.007411    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-home\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:02:49.007420 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:49.007420    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-model-cache\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:02:49.007420 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:49.007429    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-tmp-dir\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:02:49.007420 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:49.007436    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-dshm\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:02:49.007726 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:49.007444    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8prn\" (UniqueName: \"kubernetes.io/projected/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1-kube-api-access-k8prn\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:02:49.127796 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:49.127702    2569 generic.go:356] "Generic (PLEG): container finished" podID="a8b9483a-9f70-4c4b-be7b-711fd6e8dee1" containerID="48c175384c0f309b9c4cf3d33ea46b4c4e50718f1bb1b5c79252971d8ec5f812" exitCode=0
Jul 20 19:02:49.127796 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:49.127755    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"
Jul 20 19:02:49.127796 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:49.127787    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd" event={"ID":"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1","Type":"ContainerDied","Data":"48c175384c0f309b9c4cf3d33ea46b4c4e50718f1bb1b5c79252971d8ec5f812"}
Jul 20 19:02:49.128041 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:49.127826    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd" event={"ID":"a8b9483a-9f70-4c4b-be7b-711fd6e8dee1","Type":"ContainerDied","Data":"ba4ec07dd776aa77f38e32aead8f07ed3e65c992a87ceb1d14923a45a45ca01b"}
Jul 20 19:02:49.128041 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:49.127842    2569 scope.go:117] "RemoveContainer" containerID="48c175384c0f309b9c4cf3d33ea46b4c4e50718f1bb1b5c79252971d8ec5f812"
Jul 20 19:02:49.137438 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:49.137417    2569 scope.go:117] "RemoveContainer" containerID="48c175384c0f309b9c4cf3d33ea46b4c4e50718f1bb1b5c79252971d8ec5f812"
Jul 20 19:02:49.137701 ip-10-0-129-84 kubenswrapper[2569]: E0720 19:02:49.137683    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48c175384c0f309b9c4cf3d33ea46b4c4e50718f1bb1b5c79252971d8ec5f812\": container with ID starting with 48c175384c0f309b9c4cf3d33ea46b4c4e50718f1bb1b5c79252971d8ec5f812 not found: ID does not exist" containerID="48c175384c0f309b9c4cf3d33ea46b4c4e50718f1bb1b5c79252971d8ec5f812"
Jul 20 19:02:49.137752 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:49.137708    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48c175384c0f309b9c4cf3d33ea46b4c4e50718f1bb1b5c79252971d8ec5f812"} err="failed to get container status \"48c175384c0f309b9c4cf3d33ea46b4c4e50718f1bb1b5c79252971d8ec5f812\": rpc error: code = NotFound desc = could not find container \"48c175384c0f309b9c4cf3d33ea46b4c4e50718f1bb1b5c79252971d8ec5f812\": container with ID starting with 48c175384c0f309b9c4cf3d33ea46b4c4e50718f1bb1b5c79252971d8ec5f812 not found: ID does not exist"
Jul 20 19:02:49.147949 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:49.147920    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"]
Jul 20 19:02:49.152113 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:49.152084    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b7bb94/sub-test-model-b7bb94-kserve-fb448d7bc-xmjgd"]
Jul 20 19:02:49.759361 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:49.759327    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8b9483a-9f70-4c4b-be7b-711fd6e8dee1" path="/var/lib/kubelet/pods/a8b9483a-9f70-4c4b-be7b-711fd6e8dee1/volumes"
Jul 20 19:02:54.989589 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:54.989556    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:02:55.091135 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:55.091090    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:02:58.689477 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.689440    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"]
Jul 20 19:02:58.689856 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.689838    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0899eebd-4c99-41b3-9899-19a889e39aca" containerName="main"
Jul 20 19:02:58.689856 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.689851    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0899eebd-4c99-41b3-9899-19a889e39aca" containerName="main"
Jul 20 19:02:58.689929 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.689872    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8b9483a-9f70-4c4b-be7b-711fd6e8dee1" containerName="main"
Jul 20 19:02:58.689929 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.689878    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8b9483a-9f70-4c4b-be7b-711fd6e8dee1" containerName="main"
Jul 20 19:02:58.689998 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.689936    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0899eebd-4c99-41b3-9899-19a889e39aca" containerName="main"
Jul 20 19:02:58.689998 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.689948    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8b9483a-9f70-4c4b-be7b-711fd6e8dee1" containerName="main"
Jul 20 19:02:58.694218 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.694202    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.697628 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.697602    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a2f866\"/\"openshift-service-ca.crt\""
Jul 20 19:02:58.697628 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.697616    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a2f866\"/\"default-dockercfg-lckq7\""
Jul 20 19:02:58.697829 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.697617    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a2f866\"/\"rate-test-model-a2f866-kserve-self-signed-certs\""
Jul 20 19:02:58.697829 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.697610    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a2f866\"/\"kube-root-ca.crt\""
Jul 20 19:02:58.701103 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.701074    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"]
Jul 20 19:02:58.789660 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.789628    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:02:58.806544 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.806515    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-dshm\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.806701 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.806580    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-tmp-dir\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.806701 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.806607    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-tls-certs\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.806701 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.806628    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-home\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.806701 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.806688    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjhvr\" (UniqueName: \"kubernetes.io/projected/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-kube-api-access-rjhvr\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.806871 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.806736    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-model-cache\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.907323 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.907283    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-home\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.907523 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.907337    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjhvr\" (UniqueName: \"kubernetes.io/projected/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-kube-api-access-rjhvr\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.907523 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.907361    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-model-cache\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.907523 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.907402    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-dshm\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.907696 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.907585    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-tmp-dir\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.907696 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.907652    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-tls-certs\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.907808 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.907778    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-home\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.907808 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.907793    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-model-cache\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.908003 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.907979    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-tmp-dir\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.909691 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.909666    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-dshm\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.909972 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.909955    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-tls-certs\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:58.915161 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:58.915142    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjhvr\" (UniqueName: \"kubernetes.io/projected/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-kube-api-access-rjhvr\") pod \"rate-test-model-a2f866-kserve-9b7b4fd87-b74kn\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") " pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:59.005597 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:59.005566    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:02:59.145211 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:59.145186    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"]
Jul 20 19:02:59.147210 ip-10-0-129-84 kubenswrapper[2569]: W0720 19:02:59.147182    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6c2f67a_fb02_4bcf_bfa3_f648b70bf5dc.slice/crio-975921577f7e473ad3270fdd5d5b25824b82386118833b8d68fd987bdf9a8bf4 WatchSource:0}: Error finding container 975921577f7e473ad3270fdd5d5b25824b82386118833b8d68fd987bdf9a8bf4: Status 404 returned error can't find the container with id 975921577f7e473ad3270fdd5d5b25824b82386118833b8d68fd987bdf9a8bf4
Jul 20 19:02:59.180121 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:59.180085    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn" event={"ID":"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc","Type":"ContainerStarted","Data":"975921577f7e473ad3270fdd5d5b25824b82386118833b8d68fd987bdf9a8bf4"}
Jul 20 19:02:59.909450 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:02:59.909408    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:03:00.187641 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:00.187554    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn" event={"ID":"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc","Type":"ContainerStarted","Data":"342c2d0bc1ed32abff14e397a07517c96ba6b1b5ec7a01f9beccf866eeed5e17"}
Jul 20 19:03:00.205153 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:00.205095    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn" podStartSLOduration=2.20507427 podStartE2EDuration="2.20507427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:02:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:03:00.203726447 +0000 UTC m=+2977.037698840" watchObservedRunningTime="2026-07-20 19:03:00.20507427 +0000 UTC m=+2977.039046658"
Jul 20 19:03:09.005865 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:09.005824    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:03:09.006409 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:09.005878    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:03:09.018590 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:09.018565    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:03:09.251929 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:09.251893    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:03:24.108529 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:24.108496    2569 scope.go:117] "RemoveContainer" containerID="2140bb32123acdfeaba6e93955723a7a8c874adc87770aaacf9ed62735ae5c41"
Jul 20 19:03:24.117248 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:24.117225    2569 scope.go:117] "RemoveContainer" containerID="1c29aaa3262bc7773749922533ce90f122ef53d0337dbfd1785433cb3d667d0c"
Jul 20 19:03:30.406851 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:30.406740    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 19:03:30.419883 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:30.419859    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 19:03:41.855044 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:41.855006    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:03:41.955840 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:41.955803    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:03:46.616707 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:46.616665    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:03:47.470407 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:47.470371    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"]
Jul 20 19:03:47.487743 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:47.487698    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:03:48.418228 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:48.418195    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7643-140332939239856"]
Jul 20 19:03:48.421812 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:48.421796    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140332939239856"
Jul 20 19:03:48.424022 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:48.424001    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4l54m\""
Jul 20 19:03:48.427937 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:48.427911    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7643-140332939239856"]
Jul 20 19:03:48.573171 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:48.573135    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khl9g\" (UniqueName: \"kubernetes.io/projected/f54118ef-7062-4c20-a54a-0e6ec9af6844-kube-api-access-khl9g\") pod \"test-curl-7643-140332939239856\" (UID: \"f54118ef-7062-4c20-a54a-0e6ec9af6844\") " pod="opendatahub/test-curl-7643-140332939239856"
Jul 20 19:03:48.674661 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:48.674566    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khl9g\" (UniqueName: \"kubernetes.io/projected/f54118ef-7062-4c20-a54a-0e6ec9af6844-kube-api-access-khl9g\") pod \"test-curl-7643-140332939239856\" (UID: \"f54118ef-7062-4c20-a54a-0e6ec9af6844\") " pod="opendatahub/test-curl-7643-140332939239856"
Jul 20 19:03:48.682374 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:48.682340    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khl9g\" (UniqueName: \"kubernetes.io/projected/f54118ef-7062-4c20-a54a-0e6ec9af6844-kube-api-access-khl9g\") pod \"test-curl-7643-140332939239856\" (UID: \"f54118ef-7062-4c20-a54a-0e6ec9af6844\") " pod="opendatahub/test-curl-7643-140332939239856"
Jul 20 19:03:48.733247 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:48.733215    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140332939239856"
Jul 20 19:03:48.870660 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:48.870633    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7643-140332939239856"]
Jul 20 19:03:48.872917 ip-10-0-129-84 kubenswrapper[2569]: W0720 19:03:48.872891    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf54118ef_7062_4c20_a54a_0e6ec9af6844.slice/crio-fbac7ef838adf3a2e4148f9fc6e80a11fc63f4f3b4d6ea528377d1950c6a8a44 WatchSource:0}: Error finding container fbac7ef838adf3a2e4148f9fc6e80a11fc63f4f3b4d6ea528377d1950c6a8a44: Status 404 returned error can't find the container with id fbac7ef838adf3a2e4148f9fc6e80a11fc63f4f3b4d6ea528377d1950c6a8a44
Jul 20 19:03:48.874196 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:48.874178    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 19:03:49.452524 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:49.452481    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7643-140332939239856" event={"ID":"f54118ef-7062-4c20-a54a-0e6ec9af6844","Type":"ContainerStarted","Data":"fbac7ef838adf3a2e4148f9fc6e80a11fc63f4f3b4d6ea528377d1950c6a8a44"}
Jul 20 19:03:50.320812 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:50.320773    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:03:50.382264 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:50.382227    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:03:50.462254 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:50.462212    2569 generic.go:356] "Generic (PLEG): container finished" podID="f54118ef-7062-4c20-a54a-0e6ec9af6844" containerID="ff24b9dc930e9f803176407eeb7202108d80f4192e4cb8c4b27d0a44eea2841d" exitCode=0
Jul 20 19:03:50.462681 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:50.462296    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7643-140332939239856" event={"ID":"f54118ef-7062-4c20-a54a-0e6ec9af6844","Type":"ContainerDied","Data":"ff24b9dc930e9f803176407eeb7202108d80f4192e4cb8c4b27d0a44eea2841d"}
Jul 20 19:03:51.602298 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:51.602273    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140332939239856"
Jul 20 19:03:51.621382 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:51.621348    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7643-140332939239856_f54118ef-7062-4c20-a54a-0e6ec9af6844/test-curl-7643-140332939239856/0.log"
Jul 20 19:03:51.645997 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:51.645960    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7643-140332939239856"]
Jul 20 19:03:51.649368 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:51.649339    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7643-140332939239856"]
Jul 20 19:03:51.702341 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:51.702305    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khl9g\" (UniqueName: \"kubernetes.io/projected/f54118ef-7062-4c20-a54a-0e6ec9af6844-kube-api-access-khl9g\") pod \"f54118ef-7062-4c20-a54a-0e6ec9af6844\" (UID: \"f54118ef-7062-4c20-a54a-0e6ec9af6844\") "
Jul 20 19:03:51.704551 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:51.704526    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f54118ef-7062-4c20-a54a-0e6ec9af6844-kube-api-access-khl9g" (OuterVolumeSpecName: "kube-api-access-khl9g") pod "f54118ef-7062-4c20-a54a-0e6ec9af6844" (UID: "f54118ef-7062-4c20-a54a-0e6ec9af6844"). InnerVolumeSpecName "kube-api-access-khl9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 19:03:51.758597 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:51.758566    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f54118ef-7062-4c20-a54a-0e6ec9af6844" path="/var/lib/kubelet/pods/f54118ef-7062-4c20-a54a-0e6ec9af6844/volumes"
Jul 20 19:03:51.804009 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:51.803980    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khl9g\" (UniqueName: \"kubernetes.io/projected/f54118ef-7062-4c20-a54a-0e6ec9af6844-kube-api-access-khl9g\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:03:52.473940 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:52.473915    2569 scope.go:117] "RemoveContainer" containerID="ff24b9dc930e9f803176407eeb7202108d80f4192e4cb8c4b27d0a44eea2841d"
Jul 20 19:03:52.473940 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:52.473938    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140332939239856"
Jul 20 19:03:55.049320 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:55.049280    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7643-140332939346112"]
Jul 20 19:03:55.049908 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:55.049886    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f54118ef-7062-4c20-a54a-0e6ec9af6844" containerName="test-curl-7643-140332939239856"
Jul 20 19:03:55.049990 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:55.049910    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f54118ef-7062-4c20-a54a-0e6ec9af6844" containerName="test-curl-7643-140332939239856"
Jul 20 19:03:55.050100 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:55.050085    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f54118ef-7062-4c20-a54a-0e6ec9af6844" containerName="test-curl-7643-140332939239856"
Jul 20 19:03:55.054800 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:55.054779    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140332939346112"
Jul 20 19:03:55.057581 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:55.057554    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4l54m\""
Jul 20 19:03:55.059447 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:55.059419    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7643-140332939346112"]
Jul 20 19:03:55.134703 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:55.134659    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkksm\" (UniqueName: \"kubernetes.io/projected/8cd4a4d8-21a5-4e54-9384-40de421c8469-kube-api-access-tkksm\") pod \"test-curl-7643-140332939346112\" (UID: \"8cd4a4d8-21a5-4e54-9384-40de421c8469\") " pod="opendatahub/test-curl-7643-140332939346112"
Jul 20 19:03:55.235349 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:55.235304    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkksm\" (UniqueName: \"kubernetes.io/projected/8cd4a4d8-21a5-4e54-9384-40de421c8469-kube-api-access-tkksm\") pod \"test-curl-7643-140332939346112\" (UID: \"8cd4a4d8-21a5-4e54-9384-40de421c8469\") " pod="opendatahub/test-curl-7643-140332939346112"
Jul 20 19:03:55.243566 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:55.243535    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkksm\" (UniqueName: \"kubernetes.io/projected/8cd4a4d8-21a5-4e54-9384-40de421c8469-kube-api-access-tkksm\") pod \"test-curl-7643-140332939346112\" (UID: \"8cd4a4d8-21a5-4e54-9384-40de421c8469\") " pod="opendatahub/test-curl-7643-140332939346112"
Jul 20 19:03:55.365541 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:55.365445    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140332939346112"
Jul 20 19:03:55.498706 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:55.498678    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7643-140332939346112"]
Jul 20 19:03:55.500545 ip-10-0-129-84 kubenswrapper[2569]: W0720 19:03:55.500517    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8cd4a4d8_21a5_4e54_9384_40de421c8469.slice/crio-501265a25752bc08feebf6af8f74b44302075c599643e7a99543833efc559f08 WatchSource:0}: Error finding container 501265a25752bc08feebf6af8f74b44302075c599643e7a99543833efc559f08: Status 404 returned error can't find the container with id 501265a25752bc08feebf6af8f74b44302075c599643e7a99543833efc559f08
Jul 20 19:03:56.496779 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:56.496741    2569 generic.go:356] "Generic (PLEG): container finished" podID="8cd4a4d8-21a5-4e54-9384-40de421c8469" containerID="2575893d161e57a561a4641639484cb5eeec894ad948d84b6cdccb381f487192" exitCode=0
Jul 20 19:03:56.497203 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:56.496826    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7643-140332939346112" event={"ID":"8cd4a4d8-21a5-4e54-9384-40de421c8469","Type":"ContainerDied","Data":"2575893d161e57a561a4641639484cb5eeec894ad948d84b6cdccb381f487192"}
Jul 20 19:03:56.497203 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:56.496871    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7643-140332939346112" event={"ID":"8cd4a4d8-21a5-4e54-9384-40de421c8469","Type":"ContainerStarted","Data":"501265a25752bc08feebf6af8f74b44302075c599643e7a99543833efc559f08"}
Jul 20 19:03:57.639636 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:57.639611    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140332939346112"
Jul 20 19:03:57.654994 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:57.654953    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkksm\" (UniqueName: \"kubernetes.io/projected/8cd4a4d8-21a5-4e54-9384-40de421c8469-kube-api-access-tkksm\") pod \"8cd4a4d8-21a5-4e54-9384-40de421c8469\" (UID: \"8cd4a4d8-21a5-4e54-9384-40de421c8469\") "
Jul 20 19:03:57.657483 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:57.657449    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cd4a4d8-21a5-4e54-9384-40de421c8469-kube-api-access-tkksm" (OuterVolumeSpecName: "kube-api-access-tkksm") pod "8cd4a4d8-21a5-4e54-9384-40de421c8469" (UID: "8cd4a4d8-21a5-4e54-9384-40de421c8469"). InnerVolumeSpecName "kube-api-access-tkksm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 19:03:57.660620 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:57.660592    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7643-140332939346112_8cd4a4d8-21a5-4e54-9384-40de421c8469/test-curl-7643-140332939346112/0.log"
Jul 20 19:03:57.686915 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:57.686880    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7643-140332939346112"]
Jul 20 19:03:57.690292 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:57.690267    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7643-140332939346112"]
Jul 20 19:03:57.756804 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:57.756722    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkksm\" (UniqueName: \"kubernetes.io/projected/8cd4a4d8-21a5-4e54-9384-40de421c8469-kube-api-access-tkksm\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:03:57.765740 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:57.761685    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cd4a4d8-21a5-4e54-9384-40de421c8469" path="/var/lib/kubelet/pods/8cd4a4d8-21a5-4e54-9384-40de421c8469/volumes"
Jul 20 19:03:58.510875 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:58.510844    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140332939346112"
Jul 20 19:03:58.511070 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:03:58.510892    2569 scope.go:117] "RemoveContainer" containerID="2575893d161e57a561a4641639484cb5eeec894ad948d84b6cdccb381f487192"
Jul 20 19:04:02.493539 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.493469    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn" podUID="a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc" containerName="main" containerID="cri-o://342c2d0bc1ed32abff14e397a07517c96ba6b1b5ec7a01f9beccf866eeed5e17" gracePeriod=45
Jul 20 19:04:02.745894 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.745831    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:04:02.802943 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.802900    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-tls-certs\") pod \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") "
Jul 20 19:04:02.802943 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.802951    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-tmp-dir\") pod \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") "
Jul 20 19:04:02.803193 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.802974    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-model-cache\") pod \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") "
Jul 20 19:04:02.803193 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.803034    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-dshm\") pod \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") "
Jul 20 19:04:02.803193 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.803108    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-home\") pod \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") "
Jul 20 19:04:02.803193 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.803140    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjhvr\" (UniqueName: \"kubernetes.io/projected/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-kube-api-access-rjhvr\") pod \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\" (UID: \"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc\") "
Jul 20 19:04:02.803391 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.803260    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc" (UID: "a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 19:04:02.803391 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.803307    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-model-cache" (OuterVolumeSpecName: "model-cache") pod "a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc" (UID: "a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 19:04:02.803508 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.803399    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-home" (OuterVolumeSpecName: "home") pod "a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc" (UID: "a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 19:04:02.803508 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.803455    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-tmp-dir\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:04:02.803508 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.803467    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-model-cache\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:04:02.803508 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.803477    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-home\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:04:02.805246 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.805222    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc" (UID: "a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 19:04:02.805342 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.805267    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-kube-api-access-rjhvr" (OuterVolumeSpecName: "kube-api-access-rjhvr") pod "a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc" (UID: "a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc"). InnerVolumeSpecName "kube-api-access-rjhvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 19:04:02.805342 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.805302    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-dshm" (OuterVolumeSpecName: "dshm") pod "a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc" (UID: "a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 19:04:02.904324 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.904286    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjhvr\" (UniqueName: \"kubernetes.io/projected/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-kube-api-access-rjhvr\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:04:02.904324 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.904322    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-tls-certs\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:04:02.904520 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:02.904338    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc-dshm\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:04:03.068749 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.068669    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7643-140332936438816"]
Jul 20 19:04:03.069105 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.069090    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cd4a4d8-21a5-4e54-9384-40de421c8469" containerName="test-curl-7643-140332939346112"
Jul 20 19:04:03.069105 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.069106    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cd4a4d8-21a5-4e54-9384-40de421c8469" containerName="test-curl-7643-140332939346112"
Jul 20 19:04:03.069202 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.069124    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc" containerName="main"
Jul 20 19:04:03.069202 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.069130    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc" containerName="main"
Jul 20 19:04:03.069202 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.069189    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc" containerName="main"
Jul 20 19:04:03.069202 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.069198    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cd4a4d8-21a5-4e54-9384-40de421c8469" containerName="test-curl-7643-140332939346112"
Jul 20 19:04:03.072278 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.072262    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140332936438816"
Jul 20 19:04:03.075567 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.075546    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4l54m\""
Jul 20 19:04:03.079062 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.079036    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7643-140332936438816"]
Jul 20 19:04:03.106269 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.106239    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fvsw\" (UniqueName: \"kubernetes.io/projected/7c23932b-420d-409a-ba11-351c17b29394-kube-api-access-7fvsw\") pod \"test-curl-7643-140332936438816\" (UID: \"7c23932b-420d-409a-ba11-351c17b29394\") " pod="opendatahub/test-curl-7643-140332936438816"
Jul 20 19:04:03.207405 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.207371    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fvsw\" (UniqueName: \"kubernetes.io/projected/7c23932b-420d-409a-ba11-351c17b29394-kube-api-access-7fvsw\") pod \"test-curl-7643-140332936438816\" (UID: \"7c23932b-420d-409a-ba11-351c17b29394\") " pod="opendatahub/test-curl-7643-140332936438816"
Jul 20 19:04:03.215838 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.215805    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fvsw\" (UniqueName: \"kubernetes.io/projected/7c23932b-420d-409a-ba11-351c17b29394-kube-api-access-7fvsw\") pod \"test-curl-7643-140332936438816\" (UID: \"7c23932b-420d-409a-ba11-351c17b29394\") " pod="opendatahub/test-curl-7643-140332936438816"
Jul 20 19:04:03.382903 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.382826    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140332936438816"
Jul 20 19:04:03.517805 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.517781    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7643-140332936438816"]
Jul 20 19:04:03.519804 ip-10-0-129-84 kubenswrapper[2569]: W0720 19:04:03.519763    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c23932b_420d_409a_ba11_351c17b29394.slice/crio-bd8076f34c8e24ae267ccfff5931f83e70a4ece9fdaf62360214a72d57fdfd9f WatchSource:0}: Error finding container bd8076f34c8e24ae267ccfff5931f83e70a4ece9fdaf62360214a72d57fdfd9f: Status 404 returned error can't find the container with id bd8076f34c8e24ae267ccfff5931f83e70a4ece9fdaf62360214a72d57fdfd9f
Jul 20 19:04:03.540269 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.540234    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7643-140332936438816" event={"ID":"7c23932b-420d-409a-ba11-351c17b29394","Type":"ContainerStarted","Data":"bd8076f34c8e24ae267ccfff5931f83e70a4ece9fdaf62360214a72d57fdfd9f"}
Jul 20 19:04:03.541502 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.541478    2569 generic.go:356] "Generic (PLEG): container finished" podID="a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc" containerID="342c2d0bc1ed32abff14e397a07517c96ba6b1b5ec7a01f9beccf866eeed5e17" exitCode=0
Jul 20 19:04:03.541606 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.541531    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"
Jul 20 19:04:03.541606 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.541576    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn" event={"ID":"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc","Type":"ContainerDied","Data":"342c2d0bc1ed32abff14e397a07517c96ba6b1b5ec7a01f9beccf866eeed5e17"}
Jul 20 19:04:03.541696 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.541614    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn" event={"ID":"a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc","Type":"ContainerDied","Data":"975921577f7e473ad3270fdd5d5b25824b82386118833b8d68fd987bdf9a8bf4"}
Jul 20 19:04:03.541696 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.541633    2569 scope.go:117] "RemoveContainer" containerID="342c2d0bc1ed32abff14e397a07517c96ba6b1b5ec7a01f9beccf866eeed5e17"
Jul 20 19:04:03.550514 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.550496    2569 scope.go:117] "RemoveContainer" containerID="342c2d0bc1ed32abff14e397a07517c96ba6b1b5ec7a01f9beccf866eeed5e17"
Jul 20 19:04:03.550804 ip-10-0-129-84 kubenswrapper[2569]: E0720 19:04:03.550784    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"342c2d0bc1ed32abff14e397a07517c96ba6b1b5ec7a01f9beccf866eeed5e17\": container with ID starting with 342c2d0bc1ed32abff14e397a07517c96ba6b1b5ec7a01f9beccf866eeed5e17 not found: ID does not exist" containerID="342c2d0bc1ed32abff14e397a07517c96ba6b1b5ec7a01f9beccf866eeed5e17"
Jul 20 19:04:03.550873 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.550811    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342c2d0bc1ed32abff14e397a07517c96ba6b1b5ec7a01f9beccf866eeed5e17"} err="failed to get container status \"342c2d0bc1ed32abff14e397a07517c96ba6b1b5ec7a01f9beccf866eeed5e17\": rpc error: code = NotFound desc = could not find container \"342c2d0bc1ed32abff14e397a07517c96ba6b1b5ec7a01f9beccf866eeed5e17\": container with ID starting with 342c2d0bc1ed32abff14e397a07517c96ba6b1b5ec7a01f9beccf866eeed5e17 not found: ID does not exist"
Jul 20 19:04:03.562214 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.562191    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"]
Jul 20 19:04:03.564891 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.564863    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a2f866/rate-test-model-a2f866-kserve-9b7b4fd87-b74kn"]
Jul 20 19:04:03.759156 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:03.759126    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc" path="/var/lib/kubelet/pods/a6c2f67a-fb02-4bcf-bfa3-f648b70bf5dc/volumes"
Jul 20 19:04:04.549094 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:04.549042    2569 generic.go:356] "Generic (PLEG): container finished" podID="7c23932b-420d-409a-ba11-351c17b29394" containerID="a77dffef13b31977b8da0e6762228c2a80e44aa293aebbefd1b15f6782f1a1ae" exitCode=0
Jul 20 19:04:04.549498 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:04.549092    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7643-140332936438816" event={"ID":"7c23932b-420d-409a-ba11-351c17b29394","Type":"ContainerDied","Data":"a77dffef13b31977b8da0e6762228c2a80e44aa293aebbefd1b15f6782f1a1ae"}
Jul 20 19:04:05.687420 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:05.687396    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140332936438816"
Jul 20 19:04:05.707487 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:05.707458    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7643-140332936438816_7c23932b-420d-409a-ba11-351c17b29394/test-curl-7643-140332936438816/0.log"
Jul 20 19:04:05.730257 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:05.730204    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7643-140332936438816"]
Jul 20 19:04:05.731049 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:05.731029    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fvsw\" (UniqueName: \"kubernetes.io/projected/7c23932b-420d-409a-ba11-351c17b29394-kube-api-access-7fvsw\") pod \"7c23932b-420d-409a-ba11-351c17b29394\" (UID: \"7c23932b-420d-409a-ba11-351c17b29394\") "
Jul 20 19:04:05.732355 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:05.732332    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7643-140332936438816"]
Jul 20 19:04:05.733177 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:05.733156    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c23932b-420d-409a-ba11-351c17b29394-kube-api-access-7fvsw" (OuterVolumeSpecName: "kube-api-access-7fvsw") pod "7c23932b-420d-409a-ba11-351c17b29394" (UID: "7c23932b-420d-409a-ba11-351c17b29394"). InnerVolumeSpecName "kube-api-access-7fvsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 19:04:05.759507 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:05.759474    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c23932b-420d-409a-ba11-351c17b29394" path="/var/lib/kubelet/pods/7c23932b-420d-409a-ba11-351c17b29394/volumes"
Jul 20 19:04:05.832013 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:05.831916    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fvsw\" (UniqueName: \"kubernetes.io/projected/7c23932b-420d-409a-ba11-351c17b29394-kube-api-access-7fvsw\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:04:05.909558 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:05.909519    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298"]
Jul 20 19:04:05.910207 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:05.910184    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c23932b-420d-409a-ba11-351c17b29394" containerName="test-curl-7643-140332936438816"
Jul 20 19:04:05.910255 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:05.910211    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c23932b-420d-409a-ba11-351c17b29394" containerName="test-curl-7643-140332936438816"
Jul 20 19:04:05.910325 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:05.910313    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c23932b-420d-409a-ba11-351c17b29394" containerName="test-curl-7643-140332936438816"
Jul 20 19:04:05.913879 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:05.913858    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298"
Jul 20 19:04:05.916547 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:05.916517    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w8qkh\""
Jul 20 19:04:05.919735 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:05.919710    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298"]
Jul 20 19:04:05.933280 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:05.933251    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc87970e-f792-42b1-b72a-0250cab8cafd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298\" (UID: \"fc87970e-f792-42b1-b72a-0250cab8cafd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298"
Jul 20 19:04:06.034483 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:06.034442    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc87970e-f792-42b1-b72a-0250cab8cafd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298\" (UID: \"fc87970e-f792-42b1-b72a-0250cab8cafd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298"
Jul 20 19:04:06.035128 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:06.035108    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc87970e-f792-42b1-b72a-0250cab8cafd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298\" (UID: \"fc87970e-f792-42b1-b72a-0250cab8cafd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298"
Jul 20 19:04:06.227390 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:06.227292    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298"
Jul 20 19:04:06.362348 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:06.362315    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298"]
Jul 20 19:04:06.363870 ip-10-0-129-84 kubenswrapper[2569]: W0720 19:04:06.363844    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc87970e_f792_42b1_b72a_0250cab8cafd.slice/crio-c20a903992a35e5cfafb575a885ec3a377d2dec094844fca5f34132341d38bd8 WatchSource:0}: Error finding container c20a903992a35e5cfafb575a885ec3a377d2dec094844fca5f34132341d38bd8: Status 404 returned error can't find the container with id c20a903992a35e5cfafb575a885ec3a377d2dec094844fca5f34132341d38bd8
Jul 20 19:04:06.542323 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:06.542294    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:06.560849 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:06.560824    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140332936438816"
Jul 20 19:04:06.561034 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:06.560825    2569 scope.go:117] "RemoveContainer" containerID="a77dffef13b31977b8da0e6762228c2a80e44aa293aebbefd1b15f6782f1a1ae"
Jul 20 19:04:06.562771 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:06.562740    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298" event={"ID":"fc87970e-f792-42b1-b72a-0250cab8cafd","Type":"ContainerStarted","Data":"c20a903992a35e5cfafb575a885ec3a377d2dec094844fca5f34132341d38bd8"}
Jul 20 19:04:07.570517 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:07.570484    2569 generic.go:356] "Generic (PLEG): container finished" podID="fc87970e-f792-42b1-b72a-0250cab8cafd" containerID="675f2a1551ca0d9830bce1b4af13e5a98527d7a41d04115897d9626eaf274dca" exitCode=0
Jul 20 19:04:07.570893 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:07.570549    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298" event={"ID":"fc87970e-f792-42b1-b72a-0250cab8cafd","Type":"ContainerDied","Data":"675f2a1551ca0d9830bce1b4af13e5a98527d7a41d04115897d9626eaf274dca"}
Jul 20 19:04:08.190338 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:08.190301    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:08.708350 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:08.708321    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298"
Jul 20 19:04:08.760851 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:08.760826    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc87970e-f792-42b1-b72a-0250cab8cafd-maas-api-service-ca\") pod \"fc87970e-f792-42b1-b72a-0250cab8cafd\" (UID: \"fc87970e-f792-42b1-b72a-0250cab8cafd\") "
Jul 20 19:04:08.761176 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:08.761155    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc87970e-f792-42b1-b72a-0250cab8cafd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fc87970e-f792-42b1-b72a-0250cab8cafd" (UID: "fc87970e-f792-42b1-b72a-0250cab8cafd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 19:04:08.861826 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:08.861732    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc87970e-f792-42b1-b72a-0250cab8cafd-maas-api-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:04:09.581652 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:09.581618    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298" event={"ID":"fc87970e-f792-42b1-b72a-0250cab8cafd","Type":"ContainerDied","Data":"c20a903992a35e5cfafb575a885ec3a377d2dec094844fca5f34132341d38bd8"}
Jul 20 19:04:09.581831 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:09.581694    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c20a903992a35e5cfafb575a885ec3a377d2dec094844fca5f34132341d38bd8"
Jul 20 19:04:09.581831 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:09.581631    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298"
Jul 20 19:04:16.413550 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:16.413463    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"]
Jul 20 19:04:16.414174 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:16.413732    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx" podUID="4573384e-dae5-4660-80e4-3a4959d5a09f" containerName="maas-api" containerID="cri-o://2a1615eee838fdf5b8f1cf870f612b1154f87fbc7e78547c524a406326609e32" gracePeriod=30
Jul 20 19:04:16.440804 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:16.440773    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b7bb94-29742900-gd7gf"]
Jul 20 19:04:16.446390 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:16.446359    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b7bb94-29742900-gd7gf"]
Jul 20 19:04:16.448723 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:16.448691    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:16.726448 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:16.726408    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx" podUID="4573384e-dae5-4660-80e4-3a4959d5a09f" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.64:8443/health\": dial tcp 10.133.0.64:8443: connect: connection refused"
Jul 20 19:04:16.865910 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:16.865884    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"
Jul 20 19:04:16.936593 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:16.936559    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4573384e-dae5-4660-80e4-3a4959d5a09f-maas-api-tls\") pod \"4573384e-dae5-4660-80e4-3a4959d5a09f\" (UID: \"4573384e-dae5-4660-80e4-3a4959d5a09f\") "
Jul 20 19:04:16.936593 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:16.936596    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4573384e-dae5-4660-80e4-3a4959d5a09f-openshift-service-ca\") pod \"4573384e-dae5-4660-80e4-3a4959d5a09f\" (UID: \"4573384e-dae5-4660-80e4-3a4959d5a09f\") "
Jul 20 19:04:16.936861 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:16.936626    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-599c4\" (UniqueName: \"kubernetes.io/projected/4573384e-dae5-4660-80e4-3a4959d5a09f-kube-api-access-599c4\") pod \"4573384e-dae5-4660-80e4-3a4959d5a09f\" (UID: \"4573384e-dae5-4660-80e4-3a4959d5a09f\") "
Jul 20 19:04:16.937111 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:16.937084    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4573384e-dae5-4660-80e4-3a4959d5a09f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4573384e-dae5-4660-80e4-3a4959d5a09f" (UID: "4573384e-dae5-4660-80e4-3a4959d5a09f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 19:04:16.938737 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:16.938706    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4573384e-dae5-4660-80e4-3a4959d5a09f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4573384e-dae5-4660-80e4-3a4959d5a09f" (UID: "4573384e-dae5-4660-80e4-3a4959d5a09f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 19:04:16.938843 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:16.938808    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4573384e-dae5-4660-80e4-3a4959d5a09f-kube-api-access-599c4" (OuterVolumeSpecName: "kube-api-access-599c4") pod "4573384e-dae5-4660-80e4-3a4959d5a09f" (UID: "4573384e-dae5-4660-80e4-3a4959d5a09f"). InnerVolumeSpecName "kube-api-access-599c4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 19:04:17.037492 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:17.037446    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4573384e-dae5-4660-80e4-3a4959d5a09f-maas-api-tls\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:04:17.037492 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:17.037482    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4573384e-dae5-4660-80e4-3a4959d5a09f-openshift-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:04:17.037492 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:17.037492    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-599c4\" (UniqueName: \"kubernetes.io/projected/4573384e-dae5-4660-80e4-3a4959d5a09f-kube-api-access-599c4\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:04:17.623188 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:17.623148    2569 generic.go:356] "Generic (PLEG): container finished" podID="4573384e-dae5-4660-80e4-3a4959d5a09f" containerID="2a1615eee838fdf5b8f1cf870f612b1154f87fbc7e78547c524a406326609e32" exitCode=0
Jul 20 19:04:17.623651 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:17.623195    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"
Jul 20 19:04:17.623651 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:17.623234    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx" event={"ID":"4573384e-dae5-4660-80e4-3a4959d5a09f","Type":"ContainerDied","Data":"2a1615eee838fdf5b8f1cf870f612b1154f87fbc7e78547c524a406326609e32"}
Jul 20 19:04:17.623651 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:17.623288    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx" event={"ID":"4573384e-dae5-4660-80e4-3a4959d5a09f","Type":"ContainerDied","Data":"8c643ded9d694ed77ae986ebb38472ff56f7dcad8bcfa38ebff079d622631ed2"}
Jul 20 19:04:17.623651 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:17.623312    2569 scope.go:117] "RemoveContainer" containerID="2a1615eee838fdf5b8f1cf870f612b1154f87fbc7e78547c524a406326609e32"
Jul 20 19:04:17.633208 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:17.633189    2569 scope.go:117] "RemoveContainer" containerID="2a1615eee838fdf5b8f1cf870f612b1154f87fbc7e78547c524a406326609e32"
Jul 20 19:04:17.633508 ip-10-0-129-84 kubenswrapper[2569]: E0720 19:04:17.633488    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a1615eee838fdf5b8f1cf870f612b1154f87fbc7e78547c524a406326609e32\": container with ID starting with 2a1615eee838fdf5b8f1cf870f612b1154f87fbc7e78547c524a406326609e32 not found: ID does not exist" containerID="2a1615eee838fdf5b8f1cf870f612b1154f87fbc7e78547c524a406326609e32"
Jul 20 19:04:17.633559 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:17.633517    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a1615eee838fdf5b8f1cf870f612b1154f87fbc7e78547c524a406326609e32"} err="failed to get container status \"2a1615eee838fdf5b8f1cf870f612b1154f87fbc7e78547c524a406326609e32\": rpc error: code = NotFound desc = could not find container \"2a1615eee838fdf5b8f1cf870f612b1154f87fbc7e78547c524a406326609e32\": container with ID starting with 2a1615eee838fdf5b8f1cf870f612b1154f87fbc7e78547c524a406326609e32 not found: ID does not exist"
Jul 20 19:04:17.644440 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:17.644412    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"]
Jul 20 19:04:17.649768 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:17.649745    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b7bb94-6c6584dbf4-nvkfx"]
Jul 20 19:04:17.758953 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:17.758914    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38a9d02b-de71-4814-a146-d8af0319fef9" path="/var/lib/kubelet/pods/38a9d02b-de71-4814-a146-d8af0319fef9/volumes"
Jul 20 19:04:17.759346 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:17.759331    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4573384e-dae5-4660-80e4-3a4959d5a09f" path="/var/lib/kubelet/pods/4573384e-dae5-4660-80e4-3a4959d5a09f/volumes"
Jul 20 19:04:20.983555 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:20.983517    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:21.005780 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:21.005746    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298"]
Jul 20 19:04:21.008393 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:21.008368    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7bb94-7v298"]
Jul 20 19:04:21.384026 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:21.383993    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:21.758996 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:21.758963    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc87970e-f792-42b1-b72a-0250cab8cafd" path="/var/lib/kubelet/pods/fc87970e-f792-42b1-b72a-0250cab8cafd/volumes"
Jul 20 19:04:24.145634 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:24.145602    2569 scope.go:117] "RemoveContainer" containerID="8c307d04dab8e7d9d87eda4ccfc344b3ad59e660c1094574d05e93caf56e5b85"
Jul 20 19:04:27.118783 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.118746    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:27.186970 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.186933    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:27.404379 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.404301    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:27.627021 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.626988    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:27.747881 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.747848    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g"]
Jul 20 19:04:27.748316 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.748299    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc87970e-f792-42b1-b72a-0250cab8cafd" containerName="revoke-keys"
Jul 20 19:04:27.748406 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.748319    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc87970e-f792-42b1-b72a-0250cab8cafd" containerName="revoke-keys"
Jul 20 19:04:27.748406 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.748340    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4573384e-dae5-4660-80e4-3a4959d5a09f" containerName="maas-api"
Jul 20 19:04:27.748406 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.748346    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4573384e-dae5-4660-80e4-3a4959d5a09f" containerName="maas-api"
Jul 20 19:04:27.748569 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.748409    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc87970e-f792-42b1-b72a-0250cab8cafd" containerName="revoke-keys"
Jul 20 19:04:27.748569 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.748419    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4573384e-dae5-4660-80e4-3a4959d5a09f" containerName="maas-api"
Jul 20 19:04:27.754439 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.754397    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g"
Jul 20 19:04:27.756763 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.756734    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w8qkh\""
Jul 20 19:04:27.763956 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.763729    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g"]
Jul 20 19:04:27.837106 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.836905    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92d2134e-3107-41f0-8a94-2cfcef65aae8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g\" (UID: \"92d2134e-3107-41f0-8a94-2cfcef65aae8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g"
Jul 20 19:04:27.938106 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.938031    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92d2134e-3107-41f0-8a94-2cfcef65aae8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g\" (UID: \"92d2134e-3107-41f0-8a94-2cfcef65aae8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g"
Jul 20 19:04:27.938732 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:27.938707    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92d2134e-3107-41f0-8a94-2cfcef65aae8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g\" (UID: \"92d2134e-3107-41f0-8a94-2cfcef65aae8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g"
Jul 20 19:04:28.068316 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:28.068217    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g"
Jul 20 19:04:28.206606 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:28.206575    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g"]
Jul 20 19:04:28.208662 ip-10-0-129-84 kubenswrapper[2569]: W0720 19:04:28.208635    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92d2134e_3107_41f0_8a94_2cfcef65aae8.slice/crio-df68e93f8113a7852d4c56421514f32c8c726798102de9a28b65b0dd309ecd65 WatchSource:0}: Error finding container df68e93f8113a7852d4c56421514f32c8c726798102de9a28b65b0dd309ecd65: Status 404 returned error can't find the container with id df68e93f8113a7852d4c56421514f32c8c726798102de9a28b65b0dd309ecd65
Jul 20 19:04:28.305916 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:28.305881    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:28.404122 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:28.404093    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:28.684220 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:28.684122    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g" event={"ID":"92d2134e-3107-41f0-8a94-2cfcef65aae8","Type":"ContainerStarted","Data":"ec232be8b8fc43de9ecc347294bc9515a1761e1889aba9314362c0449c9e8da1"}
Jul 20 19:04:28.684220 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:28.684181    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g" event={"ID":"92d2134e-3107-41f0-8a94-2cfcef65aae8","Type":"ContainerStarted","Data":"df68e93f8113a7852d4c56421514f32c8c726798102de9a28b65b0dd309ecd65"}
Jul 20 19:04:28.699499 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:28.699439    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g" podStartSLOduration=1.6994231819999999 podStartE2EDuration="1.699423182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:04:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:04:28.697469397 +0000 UTC m=+3065.531441816" watchObservedRunningTime="2026-07-20 19:04:28.699423182 +0000 UTC m=+3065.533395571"
Jul 20 19:04:29.690681 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:29.690649    2569 generic.go:356] "Generic (PLEG): container finished" podID="92d2134e-3107-41f0-8a94-2cfcef65aae8" containerID="ec232be8b8fc43de9ecc347294bc9515a1761e1889aba9314362c0449c9e8da1" exitCode=0
Jul 20 19:04:29.691097 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:29.690736    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g" event={"ID":"92d2134e-3107-41f0-8a94-2cfcef65aae8","Type":"ContainerDied","Data":"ec232be8b8fc43de9ecc347294bc9515a1761e1889aba9314362c0449c9e8da1"}
Jul 20 19:04:30.835304 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:30.835279    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g"
Jul 20 19:04:30.966205 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:30.966126    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92d2134e-3107-41f0-8a94-2cfcef65aae8-maas-api-service-ca\") pod \"92d2134e-3107-41f0-8a94-2cfcef65aae8\" (UID: \"92d2134e-3107-41f0-8a94-2cfcef65aae8\") "
Jul 20 19:04:30.966496 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:30.966472    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92d2134e-3107-41f0-8a94-2cfcef65aae8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "92d2134e-3107-41f0-8a94-2cfcef65aae8" (UID: "92d2134e-3107-41f0-8a94-2cfcef65aae8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 19:04:31.067700 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:31.067656    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92d2134e-3107-41f0-8a94-2cfcef65aae8-maas-api-service-ca\") on node \"ip-10-0-129-84.ec2.internal\" DevicePath \"\""
Jul 20 19:04:31.701525 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:31.701492    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g"
Jul 20 19:04:31.701703 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:31.701529    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g" event={"ID":"92d2134e-3107-41f0-8a94-2cfcef65aae8","Type":"ContainerDied","Data":"df68e93f8113a7852d4c56421514f32c8c726798102de9a28b65b0dd309ecd65"}
Jul 20 19:04:31.701703 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:31.701595    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df68e93f8113a7852d4c56421514f32c8c726798102de9a28b65b0dd309ecd65"
Jul 20 19:04:38.200220 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:38.200182    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2f866-29742900-9cj45"]
Jul 20 19:04:38.201948 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:38.201917    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:38.202385 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:38.202366    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2f866-29742900-9cj45"]
Jul 20 19:04:39.758587 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:39.758555    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e771d13-5037-4d84-8200-6d2bfdd33e38" path="/var/lib/kubelet/pods/4e771d13-5037-4d84-8200-6d2bfdd33e38/volumes"
Jul 20 19:04:42.826098 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:42.826067    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:42.831360 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:42.831332    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g"]
Jul 20 19:04:42.834166 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:42.834129    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2f866-q2p4g"]
Jul 20 19:04:43.231117 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:43.231006    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:43.760572 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:43.760528    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92d2134e-3107-41f0-8a94-2cfcef65aae8" path="/var/lib/kubelet/pods/92d2134e-3107-41f0-8a94-2cfcef65aae8/volumes"
Jul 20 19:04:49.031978 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:49.031931    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:49.132540 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:49.132506    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:49.303852 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:49.303768    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:49.528242 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:49.528210    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:04:55.613609 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:55.613575    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7bf49cccc9-dnbs7_524e6c5a-090f-4d93-bd64-6564d2c72141/manager/0.log"
Jul 20 19:04:56.255817 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:56.255785    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9fdd66b9b-xk5ht_aceaa6ef-fd5c-4e3a-861c-f84e98b706b7/copy-manifests/0.log"
Jul 20 19:04:56.265228 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:56.265197    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9fdd66b9b-xk5ht_aceaa6ef-fd5c-4e3a-861c-f84e98b706b7/manager/0.log"
Jul 20 19:04:56.607631 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:56.607553    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6bc7c4f9fb-99rgq_b9e05782-6e05-475e-bf83-c4d88fc37bce/maas-api/0.log"
Jul 20 19:04:56.720035 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:56.720003    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29742870-26t84_e1ce75de-77ee-4866-a2b7-dd728e4f351a/cleanup/0.log"
Jul 20 19:04:56.836000 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:56.835975    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29742885-sdgbf_b176261d-8eea-4b2c-93b6-1aeb438cdd99/cleanup/0.log"
Jul 20 19:04:57.056930 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:57.056903    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-8rm4p_8aab38f5-c695-4462-9967-235d657071ac/postgres/0.log"
Jul 20 19:04:58.679278 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:58.679249    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-kd82c_86a0d7ee-3eaf-4cbd-8476-d485b0e85eea/kuadrant-console-plugin/0.log"
Jul 20 19:04:59.857952 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:04:59.857917    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7596f649d-zb7zz_3b5d6211-f6f0-4096-8caf-65b47857da0a/kube-auth-proxy/0.log"
Jul 20 19:05:00.200660 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:00.200573    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-vxmcq_a93d5fef-3f3b-45e6-a416-2a13e1b2286d/payload-processing/0.log"
Jul 20 19:05:00.653945 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:00.653910    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-jm6wn_3adbfffe-6de8-42a2-9379-ceda509de514/main/0.log"
Jul 20 19:05:01.115508 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:01.115479    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-czd87_6f09038b-3163-4930-b937-68ee9290ab19/main/0.log"
Jul 20 19:05:06.973588 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:06.973552    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:05:07.071023 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:07.070989    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:05:10.032751 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:10.032716    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pjhww_31d6832c-6d75-41e8-b272-7edeb41be490/global-pull-secret-syncer/0.log"
Jul 20 19:05:10.129871 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:10.129843    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log"
Jul 20 19:05:10.231456 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:10.231430    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-84.ec2.internal_409747c0910991761a210f0528443183/haproxy/0.log"
Jul 20 19:05:11.552528 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:11.552493    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-j9b5n_ec1b495c-bcb6-493b-811a-2e0957afa7a6/konnectivity-agent/0.log" limit=1024
Jul 20 19:05:15.715728 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:15.715696    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-kd82c_86a0d7ee-3eaf-4cbd-8476-d485b0e85eea/kuadrant-console-plugin/0.log"
Jul 20 19:05:17.650975 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:17.650898    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f310bdf9-6211-4281-9e13-40f481c9d5f7/alertmanager/0.log"
Jul 20 19:05:17.675544 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:17.675514    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f310bdf9-6211-4281-9e13-40f481c9d5f7/config-reloader/0.log"
Jul 20 19:05:17.698512 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:17.698482    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f310bdf9-6211-4281-9e13-40f481c9d5f7/kube-rbac-proxy-web/0.log"
Jul 20 19:05:17.721359 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:17.721334    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f310bdf9-6211-4281-9e13-40f481c9d5f7/kube-rbac-proxy/0.log"
Jul 20 19:05:17.742697 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:17.742653    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f310bdf9-6211-4281-9e13-40f481c9d5f7/kube-rbac-proxy-metric/0.log"
Jul 20 19:05:17.764391 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:17.764369    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f310bdf9-6211-4281-9e13-40f481c9d5f7/prom-label-proxy/0.log"
Jul 20 19:05:17.789504 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:17.789476    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f310bdf9-6211-4281-9e13-40f481c9d5f7/init-config-reloader/0.log"
Jul 20 19:05:17.997700 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:17.997669    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-m22pt_c556e218-7633-4bd3-91c3-f1979da2efec/monitoring-plugin/0.log"
Jul 20 19:05:18.027839 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.027810    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-299wt_7d6e0182-444e-4a6c-bf6a-c76b947418d6/node-exporter/0.log"
Jul 20 19:05:18.050300 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.050270    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-299wt_7d6e0182-444e-4a6c-bf6a-c76b947418d6/kube-rbac-proxy/0.log"
Jul 20 19:05:18.073139 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.073114    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-299wt_7d6e0182-444e-4a6c-bf6a-c76b947418d6/init-textfile/0.log"
Jul 20 19:05:18.336816 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.336744    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_02e37e7f-933e-49bf-9935-90f64fdf43a1/prometheus/0.log"
Jul 20 19:05:18.358640 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.358605    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_02e37e7f-933e-49bf-9935-90f64fdf43a1/config-reloader/0.log"
Jul 20 19:05:18.379514 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.379488    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_02e37e7f-933e-49bf-9935-90f64fdf43a1/thanos-sidecar/0.log"
Jul 20 19:05:18.400650 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.400624    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_02e37e7f-933e-49bf-9935-90f64fdf43a1/kube-rbac-proxy-web/0.log"
Jul 20 19:05:18.421511 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.421487    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_02e37e7f-933e-49bf-9935-90f64fdf43a1/kube-rbac-proxy/0.log"
Jul 20 19:05:18.444186 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.444155    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_02e37e7f-933e-49bf-9935-90f64fdf43a1/kube-rbac-proxy-thanos/0.log"
Jul 20 19:05:18.464449 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.464422    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_02e37e7f-933e-49bf-9935-90f64fdf43a1/init-config-reloader/0.log"
Jul 20 19:05:18.543367 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.543333    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-9t862_d9c8c973-4cfe-4a11-bbe0-f2a734ef89a3/prometheus-operator-admission-webhook/0.log"
Jul 20 19:05:18.574160 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.574115    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-564f949944-bc7cx_04484408-c34f-4c9e-8efe-48871f99c4f7/telemeter-client/0.log"
Jul 20 19:05:18.597176 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.597089    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-564f949944-bc7cx_04484408-c34f-4c9e-8efe-48871f99c4f7/reload/0.log"
Jul 20 19:05:18.621463 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.621436    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-564f949944-bc7cx_04484408-c34f-4c9e-8efe-48871f99c4f7/kube-rbac-proxy/0.log"
Jul 20 19:05:18.664126 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.664097    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564d88c9cb-dm42m_1dc72865-f206-4b99-b0b8-6871c20bb36f/thanos-query/0.log"
Jul 20 19:05:18.686701 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.686671    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564d88c9cb-dm42m_1dc72865-f206-4b99-b0b8-6871c20bb36f/kube-rbac-proxy-web/0.log"
Jul 20 19:05:18.717094 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.717046    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564d88c9cb-dm42m_1dc72865-f206-4b99-b0b8-6871c20bb36f/kube-rbac-proxy/0.log"
Jul 20 19:05:18.739133 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.739104    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564d88c9cb-dm42m_1dc72865-f206-4b99-b0b8-6871c20bb36f/prom-label-proxy/0.log"
Jul 20 19:05:18.761028 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.761002    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564d88c9cb-dm42m_1dc72865-f206-4b99-b0b8-6871c20bb36f/kube-rbac-proxy-rules/0.log"
Jul 20 19:05:18.782395 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:18.782365    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564d88c9cb-dm42m_1dc72865-f206-4b99-b0b8-6871c20bb36f/kube-rbac-proxy-metrics/0.log"
Jul 20 19:05:20.246825 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.246786    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"]
Jul 20 19:05:20.247449 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.247422    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92d2134e-3107-41f0-8a94-2cfcef65aae8" containerName="revoke-keys"
Jul 20 19:05:20.247449 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.247443    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="92d2134e-3107-41f0-8a94-2cfcef65aae8" containerName="revoke-keys"
Jul 20 19:05:20.247591 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.247569    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="92d2134e-3107-41f0-8a94-2cfcef65aae8" containerName="revoke-keys"
Jul 20 19:05:20.250714 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.250693    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.252974 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.252950    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wtj6n\"/\"openshift-service-ca.crt\""
Jul 20 19:05:20.253787 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.253770    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wtj6n\"/\"default-dockercfg-pl49w\""
Jul 20 19:05:20.253868 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.253799    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wtj6n\"/\"kube-root-ca.crt\""
Jul 20 19:05:20.260441 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.260413    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"]
Jul 20 19:05:20.343304 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.343264    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f3ca44b-40dd-46e0-8d96-470fd8d75525-lib-modules\") pod \"perf-node-gather-daemonset-rfc7f\" (UID: \"0f3ca44b-40dd-46e0-8d96-470fd8d75525\") " pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.343514 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.343314    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f3ca44b-40dd-46e0-8d96-470fd8d75525-sys\") pod \"perf-node-gather-daemonset-rfc7f\" (UID: \"0f3ca44b-40dd-46e0-8d96-470fd8d75525\") " pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.343514 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.343403    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wffwl\" (UniqueName: \"kubernetes.io/projected/0f3ca44b-40dd-46e0-8d96-470fd8d75525-kube-api-access-wffwl\") pod \"perf-node-gather-daemonset-rfc7f\" (UID: \"0f3ca44b-40dd-46e0-8d96-470fd8d75525\") " pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.343514 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.343426    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0f3ca44b-40dd-46e0-8d96-470fd8d75525-proc\") pod \"perf-node-gather-daemonset-rfc7f\" (UID: \"0f3ca44b-40dd-46e0-8d96-470fd8d75525\") " pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.343640 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.343539    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0f3ca44b-40dd-46e0-8d96-470fd8d75525-podres\") pod \"perf-node-gather-daemonset-rfc7f\" (UID: \"0f3ca44b-40dd-46e0-8d96-470fd8d75525\") " pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.444831 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.444796    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0f3ca44b-40dd-46e0-8d96-470fd8d75525-proc\") pod \"perf-node-gather-daemonset-rfc7f\" (UID: \"0f3ca44b-40dd-46e0-8d96-470fd8d75525\") " pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.445038 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.444859    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0f3ca44b-40dd-46e0-8d96-470fd8d75525-podres\") pod \"perf-node-gather-daemonset-rfc7f\" (UID: \"0f3ca44b-40dd-46e0-8d96-470fd8d75525\") " pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.445038 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.444888    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f3ca44b-40dd-46e0-8d96-470fd8d75525-lib-modules\") pod \"perf-node-gather-daemonset-rfc7f\" (UID: \"0f3ca44b-40dd-46e0-8d96-470fd8d75525\") " pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.445038 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.444907    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f3ca44b-40dd-46e0-8d96-470fd8d75525-sys\") pod \"perf-node-gather-daemonset-rfc7f\" (UID: \"0f3ca44b-40dd-46e0-8d96-470fd8d75525\") " pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.445038 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.444918    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0f3ca44b-40dd-46e0-8d96-470fd8d75525-proc\") pod \"perf-node-gather-daemonset-rfc7f\" (UID: \"0f3ca44b-40dd-46e0-8d96-470fd8d75525\") " pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.445038 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.444966    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wffwl\" (UniqueName: \"kubernetes.io/projected/0f3ca44b-40dd-46e0-8d96-470fd8d75525-kube-api-access-wffwl\") pod \"perf-node-gather-daemonset-rfc7f\" (UID: \"0f3ca44b-40dd-46e0-8d96-470fd8d75525\") " pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.445285 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.445042    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f3ca44b-40dd-46e0-8d96-470fd8d75525-lib-modules\") pod \"perf-node-gather-daemonset-rfc7f\" (UID: \"0f3ca44b-40dd-46e0-8d96-470fd8d75525\") " pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.445285 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.445047    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0f3ca44b-40dd-46e0-8d96-470fd8d75525-podres\") pod \"perf-node-gather-daemonset-rfc7f\" (UID: \"0f3ca44b-40dd-46e0-8d96-470fd8d75525\") " pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.445285 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.445109    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f3ca44b-40dd-46e0-8d96-470fd8d75525-sys\") pod \"perf-node-gather-daemonset-rfc7f\" (UID: \"0f3ca44b-40dd-46e0-8d96-470fd8d75525\") " pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.452646 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.452616    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wffwl\" (UniqueName: \"kubernetes.io/projected/0f3ca44b-40dd-46e0-8d96-470fd8d75525-kube-api-access-wffwl\") pod \"perf-node-gather-daemonset-rfc7f\" (UID: \"0f3ca44b-40dd-46e0-8d96-470fd8d75525\") " pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.548119 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.547978    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/2.log"
Jul 20 19:05:20.552348 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.552328    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pb2nw_c4f56323-217f-4fe1-be14-ec9ad4b7b81a/console-operator/3.log"
Jul 20 19:05:20.561463 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.561446    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.692780 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.692750    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"]
Jul 20 19:05:20.694461 ip-10-0-129-84 kubenswrapper[2569]: W0720 19:05:20.694436    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0f3ca44b_40dd_46e0_8d96_470fd8d75525.slice/crio-e6772d9a64a4b1a90e3de5c80f2fb3f0a33a77b1955cf06f4faa69449fd5a93f WatchSource:0}: Error finding container e6772d9a64a4b1a90e3de5c80f2fb3f0a33a77b1955cf06f4faa69449fd5a93f: Status 404 returned error can't find the container with id e6772d9a64a4b1a90e3de5c80f2fb3f0a33a77b1955cf06f4faa69449fd5a93f
Jul 20 19:05:20.962634 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.962540    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f" event={"ID":"0f3ca44b-40dd-46e0-8d96-470fd8d75525","Type":"ContainerStarted","Data":"88993a5bcd686bf247f8339f9696bc13c5e353167ca3ba537c0e414f654f306b"}
Jul 20 19:05:20.962634 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.962589    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f" event={"ID":"0f3ca44b-40dd-46e0-8d96-470fd8d75525","Type":"ContainerStarted","Data":"e6772d9a64a4b1a90e3de5c80f2fb3f0a33a77b1955cf06f4faa69449fd5a93f"}
Jul 20 19:05:20.962858 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.962659    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:20.977503 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:20.977452    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f" podStartSLOduration=0.977435372 podStartE2EDuration="977.435372ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:05:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:05:20.977009373 +0000 UTC m=+3117.810981763" watchObservedRunningTime="2026-07-20 19:05:20.977435372 +0000 UTC m=+3117.811407760"
Jul 20 19:05:21.107613 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:21.107582    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-tvt44_ad8acde9-8e96-4934-be94-7c928abaa587/download-server/0.log"
Jul 20 19:05:22.711368 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:22.711336    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-85nj6_d223e041-067c-49a8-a19c-b4879963bc0e/dns/0.log"
Jul 20 19:05:22.732773 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:22.732747    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-85nj6_d223e041-067c-49a8-a19c-b4879963bc0e/kube-rbac-proxy/0.log"
Jul 20 19:05:22.827608 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:22.827566    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8bf9p_17e7c5f3-421c-4bdb-9e4e-67586149b43d/dns-node-resolver/0.log"
Jul 20 19:05:23.460234 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:23.460200    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lkqb9_d5a1608a-991c-4c92-978f-7f00e84fda46/node-ca/0.log"
Jul 20 19:05:24.577993 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:24.577965    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7596f649d-zb7zz_3b5d6211-f6f0-4096-8caf-65b47857da0a/kube-auth-proxy/0.log"
Jul 20 19:05:24.692401 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:24.692361    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-vxmcq_a93d5fef-3f3b-45e6-a416-2a13e1b2286d/payload-processing/0.log"
Jul 20 19:05:25.393375 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:25.393341    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5n5pt_fb74a816-b89d-4e4d-b322-cc029854d831/serve-healthcheck-canary/0.log"
Jul 20 19:05:25.898748 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:25.898715    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-t7pmk_162cbe48-6490-47b2-b881-85d8e8e95e19/insights-operator/0.log"
Jul 20 19:05:25.900119 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:25.900102    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-t7pmk_162cbe48-6490-47b2-b881-85d8e8e95e19/insights-operator/1.log"
Jul 20 19:05:26.058171 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:26.058140    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7q9t5_629056e9-18ee-4294-a8ab-42081cff41b4/kube-rbac-proxy/0.log"
Jul 20 19:05:26.079227 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:26.079195    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7q9t5_629056e9-18ee-4294-a8ab-42081cff41b4/exporter/0.log"
Jul 20 19:05:26.100469 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:26.100433    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7q9t5_629056e9-18ee-4294-a8ab-42081cff41b4/extractor/0.log"
Jul 20 19:05:26.978557 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:26.978529    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wtj6n/perf-node-gather-daemonset-rfc7f"
Jul 20 19:05:28.642306 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:28.642273    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7bf49cccc9-dnbs7_524e6c5a-090f-4d93-bd64-6564d2c72141/manager/0.log"
Jul 20 19:05:29.007454 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:29.007415    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9fdd66b9b-xk5ht_aceaa6ef-fd5c-4e3a-861c-f84e98b706b7/manager/0.log"
Jul 20 19:05:29.029035 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:29.029005    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9fdd66b9b-xk5ht_aceaa6ef-fd5c-4e3a-861c-f84e98b706b7/copy-manifests/0.log"
Jul 20 19:05:30.464546 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:30.464521    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-bdf467c7d-vtdk8_d8e4a50a-e606-41d7-a4aa-df23be7712de/manager/0.log"
Jul 20 19:05:30.509068 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:30.509011    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-7b28t_fd8a6550-c238-435f-aedb-0b8a5159f9f6/openshift-lws-operator/0.log"
Jul 20 19:05:30.768975 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:30.768946    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7596f649d-zb7zz_3b5d6211-f6f0-4096-8caf-65b47857da0a/kube-auth-proxy/0.log"
Jul 20 19:05:30.859902 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:30.859871    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-vxmcq_a93d5fef-3f3b-45e6-a416-2a13e1b2286d/payload-processing/0.log"
Jul 20 19:05:32.217695 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:32.217668    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7596f649d-zb7zz_3b5d6211-f6f0-4096-8caf-65b47857da0a/kube-auth-proxy/0.log"
Jul 20 19:05:32.311841 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:32.311813    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-vxmcq_a93d5fef-3f3b-45e6-a416-2a13e1b2286d/payload-processing/0.log"
Jul 20 19:05:35.279471 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:35.279436    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-w9t2q_f43ff13a-ead5-45d1-9e43-b17ddef43cf5/migrator/0.log"
Jul 20 19:05:35.302250 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:35.302222    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-w9t2q_f43ff13a-ead5-45d1-9e43-b17ddef43cf5/graceful-termination/0.log"
Jul 20 19:05:36.986211 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:36.986136    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pq7xn_f2c72dbe-b803-492b-9046-eeeacf69b9cf/kube-multus-additional-cni-plugins/0.log"
Jul 20 19:05:37.008633 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:37.008601    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pq7xn_f2c72dbe-b803-492b-9046-eeeacf69b9cf/egress-router-binary-copy/0.log"
Jul 20 19:05:37.029569 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:37.029541    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pq7xn_f2c72dbe-b803-492b-9046-eeeacf69b9cf/cni-plugins/0.log"
Jul 20 19:05:37.054368 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:37.054347    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pq7xn_f2c72dbe-b803-492b-9046-eeeacf69b9cf/bond-cni-plugin/0.log"
Jul 20 19:05:37.078583 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:37.078556    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pq7xn_f2c72dbe-b803-492b-9046-eeeacf69b9cf/routeoverride-cni/0.log"
Jul 20 19:05:37.099980 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:37.099954    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pq7xn_f2c72dbe-b803-492b-9046-eeeacf69b9cf/whereabouts-cni-bincopy/0.log"
Jul 20 19:05:37.120582 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:37.120556    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pq7xn_f2c72dbe-b803-492b-9046-eeeacf69b9cf/whereabouts-cni/0.log"
Jul 20 19:05:37.203866 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:37.203835    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vc9wm_6d755879-19d5-463b-bd98-83a890d00721/kube-multus/0.log"
Jul 20 19:05:37.312719 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:37.312686    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-g4z7w_1bae9e54-a7d1-46ba-9bb6-9602c8d53962/network-metrics-daemon/0.log"
Jul 20 19:05:37.336533 ip-10-0-129-84 kubenswrapper[2569]: I0720 19:05:37.336506    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-g4z7w_1bae9e54-a7d1-46ba-9bb6-9602c8d53962/kube-rbac-proxy/0.log"