Jul 09 13:56:06.976289 ip-10-0-128-186 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 13:56:07.408326 ip-10-0-128-186 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 13:56:07.408326 ip-10-0-128-186 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 13:56:07.408326 ip-10-0-128-186 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 13:56:07.408326 ip-10-0-128-186 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 13:56:07.408326 ip-10-0-128-186 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 13:56:07.411355 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.411265    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 13:56:07.417862 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417840    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 13:56:07.417862 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417857    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 13:56:07.417862 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417861    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 13:56:07.417862 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417864    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 13:56:07.417862 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417868    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417871    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417874    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417878    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417882    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417885    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417888    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417890    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417893    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417896    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417898    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417901    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417903    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417905    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417908    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417911    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417913    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417916    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417918    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 13:56:07.418075 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417921    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417923    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417925    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417928    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417937    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417940    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417942    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417945    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417947    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417949    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417952    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417954    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417956    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417959    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417961    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417964    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417967    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417969    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417972    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417988    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 13:56:07.418532 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417991    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417994    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417996    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.417999    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418001    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418003    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418006    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418008    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418011    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418015    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418017    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418020    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418022    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418024    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418027    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418030    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418032    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418035    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418037    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418040    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 13:56:07.419053 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418042    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418044    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418047    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418049    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418052    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418055    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418057    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418062    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418064    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418067    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418069    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418072    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418075    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418078    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418083    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418087    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418090    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418093    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418095    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 13:56:07.419527 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418098    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418101    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418104    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418106    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418109    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418111    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418115    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418119    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418122    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418125    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418128    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418131    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418133    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418136    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418138    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418141    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418143    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418838    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418844    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 13:56:07.419965 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418847    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418851    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418854    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418857    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418859    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418862    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418864    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418867    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418869    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418872    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418874    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418877    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418879    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418882    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418884    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418889    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418893    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418896    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418899    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 13:56:07.420431 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418902    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418904    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418907    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418909    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418912    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418915    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418918    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418920    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418923    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418925    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418927    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418930    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418933    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418935    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418938    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418941    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418944    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418946    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418949    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418951    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 13:56:07.420899 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418953    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418956    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418958    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418961    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418963    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418965    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418968    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418970    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418973    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418991    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418994    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418997    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.418999    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419002    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419004    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419007    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419009    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419012    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419014    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419016    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 13:56:07.421415 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419019    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419021    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419024    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419026    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419029    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419031    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419036    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419039    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419043    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419046    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419050    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419054    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419057    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419059    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419063    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419066    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419069    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419071    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419074    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 13:56:07.421939 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419076    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419078    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419081    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419084    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419086    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419089    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419092    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419095    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419097    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419100    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419102    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419104    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419107    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419109    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419112    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419114    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419117    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419119    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419122    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419489    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 13:56:07.422433 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419493    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419496    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419498    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419501    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419504    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419506    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419509    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419511    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419514    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419516    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419519    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419522    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419525    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419529    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419532    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419536    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419538    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419541    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419543    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 13:56:07.422915 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419546    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419548    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419551    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419556    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419558    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419561    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419563    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419566    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419569    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419571    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419574    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419579    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419581    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419584    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419586    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419589    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419592    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419594    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419597    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 13:56:07.423386 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419599    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419602    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419604    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419607    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419609    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419612    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419614    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419616    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419619    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419621    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419624    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419627    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419629    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419632    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419634    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419637    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419639    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419642    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419644    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419646    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 13:56:07.423841 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419649    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419651    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419653    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419656    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419660    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419662    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419665    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419669    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419673    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419677    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419679    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419682    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419684    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419687    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419689    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419691    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419694    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419696    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419699    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419701    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 13:56:07.424321 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419703    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419706    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419708    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419711    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419713    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419716    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419718    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419721    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419723    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419725    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419728    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419730    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419733    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419735    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419737    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419740    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419744    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419746    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419749    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.419751    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 13:56:07.424889 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419832    2579 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419848    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419858    2579 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419864    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419870    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419875    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419880    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419884    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419887    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419890    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419893    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419896    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419899    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419902    2579 flags.go:64] FLAG: --cgroup-root=""
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419905    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419907    2579 flags.go:64] FLAG: --client-ca-file=""
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419911    2579 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419914    2579 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419918    2579 flags.go:64] FLAG: --cluster-domain=""
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419921    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419924    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419927    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419951    2579 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419956    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 13:56:07.425399 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419959    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419962    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419966    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419969    2579 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419987    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419990    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419993    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.419996    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420000    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420003    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420006    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420009    2579 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420012    2579 flags.go:64] FLAG: --enable-server="true"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420014    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420019    2579 flags.go:64] FLAG: --event-burst="100"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420022    2579 flags.go:64] FLAG: --event-qps="50"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420024    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420027    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420030    2579 flags.go:64] FLAG: --eviction-hard=""
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420034    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420037    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420039    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420042    2579 flags.go:64] FLAG: --eviction-soft=""
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420045    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420048    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 13:56:07.426036 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420052    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420054    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420059    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420062    2579 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420064    2579 flags.go:64] FLAG: --feature-gates=""
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420068    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420071    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420074    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420077    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420080    2579 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420082    2579 flags.go:64] FLAG: --help="false"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420086    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-128-186.ec2.internal"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420089    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420092    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420094    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420098    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420101    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420104    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420106    2579 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420109    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420111    2579 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420114    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420117    2579 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420120    2579 flags.go:64] FLAG: --kube-reserved=""
Jul 09 13:56:07.426694 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420122    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420125    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420128    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420130    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420133    2579 flags.go:64] FLAG: --lock-file=""
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420135    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420138    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420141    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420146    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420150    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420154    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420157    2579 flags.go:64] FLAG: --logging-format="text"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420160    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420163    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420165    2579 flags.go:64] FLAG: --manifest-url=""
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420168    2579 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420175    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420177    2579 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420181    2579 flags.go:64] FLAG: --max-pods="110"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420184    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420187    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420190    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420192    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420195    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420198    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 13:56:07.427341 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420201    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420208    2579 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420211    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420214    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420222    2579 flags.go:64] FLAG: --pod-cidr=""
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420226    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420232    2579 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420235    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420239    2579 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420241    2579 flags.go:64] FLAG: --port="10250"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420244    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420247    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04032151340a31725"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420250    2579 flags.go:64] FLAG: --qos-reserved=""
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420253    2579 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420256    2579 flags.go:64] FLAG: --register-node="true"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420258    2579 flags.go:64] FLAG: --register-with-taints=""
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420262    2579 flags.go:64] FLAG: --registry-burst="10"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420267    2579 flags.go:64] FLAG: --registry-qps="5"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420269    2579 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420272    2579 flags.go:64] FLAG: --reserved-memory=""
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420275    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420278    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420281    2579 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420284    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420286    2579 flags.go:64] FLAG: --runonce="false"
Jul 09 13:56:07.427946 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420289    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420292    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420295    2579 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420297    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420300    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420303    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420306    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420309    2579 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420311    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420314    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420317    2579 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420319    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420322    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420325    2579 flags.go:64] FLAG: --system-cgroups=""
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420327    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420333    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420335    2579 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420338    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420342    2579 flags.go:64] FLAG: --tls-min-version=""
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420344    2579 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420346    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420349    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420352    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420354    2579 flags.go:64] FLAG: --v="2"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420359    2579 flags.go:64] FLAG: --version="false"
Jul 09 13:56:07.428565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420363    2579 flags.go:64] FLAG: --vmodule=""
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420367    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.420370    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420479    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420483    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420485    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420488    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420491    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420493    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420496    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420499    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420502    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420505    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420508    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420510    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420513    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420515    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420518    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420521    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420523    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 13:56:07.429193 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420526    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420528    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420530    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420533    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420535    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420538    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420540    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420542    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420545    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420548    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420550    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420554    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420556    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420559    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420562    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420565    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420567    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420571    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420575    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420578    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 13:56:07.429670 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420580    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420584    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420586    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420589    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420591    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420594    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420597    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420599    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420602    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420604    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420606    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420609    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420611    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420614    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420618    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420621    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420624    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420627    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420630    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 13:56:07.430249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420633    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420636    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420638    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420641    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420644    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420647    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420649    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420652    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420655    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420657    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420659    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420662    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420664    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420667    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420670    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420673    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420675    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420678    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420680    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420683    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 13:56:07.430706 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420685    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420688    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420690    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420693    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420695    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420698    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420700    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420703    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420705    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420708    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420710    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420712    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420715    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420717    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420720    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420722    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420726    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420729    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420732    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420734    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 13:56:07.431241 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420737    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 13:56:07.431732 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420739    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 13:56:07.431732 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.420742    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 13:56:07.431732 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.421347    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 13:56:07.431732 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.428387    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 09 13:56:07.431732 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.428401    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 13:56:07.431732 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.428421    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 13:56:07.431732 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.428427    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 13:56:07.431732 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428480    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 13:56:07.431732 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428486    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 13:56:07.431732 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428490    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 13:56:07.431732 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428493    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 13:56:07.431732 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428496    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 13:56:07.431732 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428499    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 13:56:07.431732 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428502    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428504    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428507    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428509    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428512    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428515    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428517    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428520    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428522    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428524    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428527    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428530    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428532    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428535    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428537    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428540    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428543    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428545    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428548    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428551    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 13:56:07.432093 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428553    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428556    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428559    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428562    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428565    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428568    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428570    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428573    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428575    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428578    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428581    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428583    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428585    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428588    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428591    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428595    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428598    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428601    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428604    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428607    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 13:56:07.432606 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428610    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428612    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428615    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428618    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428621    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428623    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428626    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428629    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428631    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428634    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428636    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428639    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428643    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428646    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428649    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428652    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428655    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428658    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428660    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 13:56:07.433105 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428662    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428665    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428667    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428670    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428672    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428675    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428677    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428680    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428683    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428685    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428687    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428690    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428692    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428695    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428698    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428700    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428702    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428705    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428707    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428709    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 13:56:07.433564 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428712    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 13:56:07.434195 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428715    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 13:56:07.434195 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428717    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 13:56:07.434195 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428720    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 13:56:07.434195 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428722    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 13:56:07.434195 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428724    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 13:56:07.434195 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428727    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 13:56:07.434195 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428729    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 13:56:07.434195 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428733    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 13:56:07.434195 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428735    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 13:56:07.434195 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428741    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 13:56:07.434195 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428744    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 13:56:07.434195 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428746    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 13:56:07.434195 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428748    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 13:56:07.434195 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.428753    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 13:56:07.434195 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428863    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428868    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428871    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428874    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428877    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428879    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428882    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428884    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428887    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428890    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428892    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428895    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428897    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428900    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428902    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428906    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428909    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428912    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428915    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428917    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 13:56:07.434597 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428920    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428922    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428925    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428927    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428930    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428932    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428935    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428937    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428942    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428946    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428949    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428952    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428955    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428957    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428960    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428962    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428964    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428966    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428969    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 13:56:07.435128 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428971    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428974    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428991    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428993    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428995    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.428998    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429000    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429002    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429005    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429008    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429010    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429012    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429015    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429017    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429019    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429022    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429024    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429026    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429029    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429031    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 13:56:07.435576 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429034    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429036    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429040    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429043    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429045    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429047    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429050    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429052    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429054    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429057    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429059    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429061    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429064    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429066    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429068    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429070    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429073    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429075    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429077    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429080    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 13:56:07.436123 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429082    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429085    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429088    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429090    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429092    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429095    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429097    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429099    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429102    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429105    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429107    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429109    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429112    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429114    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429117    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429120    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429123    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429125    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429127    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 13:56:07.436626 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:07.429130    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 13:56:07.437129 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.429134    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 13:56:07.437129 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.429230    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 13:56:07.437129 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.432094    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 13:56:07.437129 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.433845    2579 server.go:1019] "Starting client certificate rotation"
Jul 09 13:56:07.437129 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.433954    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 13:56:07.437129 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.434021    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 13:56:07.437129 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.436439    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 13:56:07.441509 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.441491    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 13:56:07.448202 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.448181    2579 log.go:25] "Validated CRI v1 runtime API"
Jul 09 13:56:07.450157 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.450144    2579 log.go:25] "Validated CRI v1 image API"
Jul 09 13:56:07.450877 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.450863    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 13:56:07.457061 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.457044    2579 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b4eaef11-bafb-4059-a1db-8df44bcdc4b1:/dev/nvme0n1p4 db0243bc-57d1-4b44-912c-11ddd6e7891e:/dev/nvme0n1p3]
Jul 09 13:56:07.457123 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.457061    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 13:56:07.462059 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.461938    2579 manager.go:216] Machine: {Timestamp:2026-07-09 13:56:07.460673368 +0000 UTC m=+0.357817935 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3091326 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec256af39f0aa5277d7a043226c85ffc SystemUUID:ec256af3-9f0a-a527-7d7a-043226c85ffc BootID:5c1df6ee-b4bd-4163-a1d9-3fe7812403e2 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6b:39:6c:00:03 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6b:39:6c:00:03 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:48:06:83:13:d9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 13:56:07.462059 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.462058    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 13:56:07.462170 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.462159    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 13:56:07.465203 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.465175    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 13:56:07.465338 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.465205    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-186.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 13:56:07.466526 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.466516    2579 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 13:56:07.466564 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.466528    2579 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 13:56:07.466564 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.466536    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 13:56:07.467440 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.467428    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 13:56:07.468797 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.468786    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 13:56:07.471573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.471558    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 13:56:07.472048 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.472037    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 13:56:07.472153 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.472144    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 13:56:07.472193 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.472172    2579 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 13:56:07.472225 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.472196    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 13:56:07.472225 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.472208    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 13:56:07.472225 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.472216    2579 kubelet.go:398] "Adding apiserver pod source"
Jul 09 13:56:07.472225 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.472225    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 13:56:07.472626 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.472615    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 13:56:07.472660 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.472632    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 13:56:07.474904 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.474886    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 09 13:56:07.476825 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.476813    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 13:56:07.476897 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.476842    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 13:56:07.478595 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.478580    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 13:56:07.478595 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.478598    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 13:56:07.478677 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.478604    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 13:56:07.478677 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.478610    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 13:56:07.478677 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.478633    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 13:56:07.478677 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.478639    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 13:56:07.478677 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.478647    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 13:56:07.478677 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.478652    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 13:56:07.478677 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.478661    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 13:56:07.478677 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.478666    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 13:56:07.478677 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.478674    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 13:56:07.478677 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.478679    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 13:56:07.480207 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.480185    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 13:56:07.480207 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.480198    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 13:56:07.483426 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.483071    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-186.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 13:56:07.483578 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.483537    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-186.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 13:56:07.483722 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.483710    2579 server.go:1268] "Started kubelet"
Jul 09 13:56:07.484165 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.484137    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 13:56:07.484234 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.484198    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 13:56:07.484610 ip-10-0-128-186 systemd[1]: Started Kubernetes Kubelet.
Jul 09 13:56:07.484925 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.484850    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 13:56:07.485014 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.484931    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 13:56:07.486170 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.486155    2579 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 13:56:07.486253 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.486224    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 13:56:07.490858 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.490837    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 13:56:07.492577 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.492557    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 13:56:07.492662 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.492580    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 13:56:07.492662 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.492625    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:07.493619 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.492208    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-186.ec2.internal.18c0a3a1bc5872f1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-186.ec2.internal,UID:ip-10-0-128-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-186.ec2.internal,},FirstTimestamp:2026-07-09 13:56:07.483691761 +0000 UTC m=+0.380836328,LastTimestamp:2026-07-09 13:56:07.483691761 +0000 UTC m=+0.380836328,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-186.ec2.internal,}"
Jul 09 13:56:07.493619 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.493433    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 13:56:07.493758 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.493631    2579 factory.go:55] Registering systemd factory
Jul 09 13:56:07.493758 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.493643    2579 factory.go:223] Registration of the systemd container factory successfully
Jul 09 13:56:07.493864 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.493851    2579 factory.go:153] Registering CRI-O factory
Jul 09 13:56:07.493917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.493869    2579 factory.go:223] Registration of the crio container factory successfully
Jul 09 13:56:07.493917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.493893    2579 factory.go:103] Registering Raw factory
Jul 09 13:56:07.493917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.493910    2579 manager.go:1196] Started watching for new ooms in manager
Jul 09 13:56:07.494265 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.494252    2579 manager.go:349] Starting recovery of all containers
Jul 09 13:56:07.494503 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.494487    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 13:56:07.494561 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.494487    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 13:56:07.494561 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.494518    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 13:56:07.494660 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.494618    2579 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 13:56:07.494660 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.494626    2579 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 13:56:07.495224 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.495203    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xrh8x"
Jul 09 13:56:07.502607 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.502552    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 13:56:07.503944 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.503919    2579 manager.go:354] Recovery completed
Jul 09 13:56:07.504560 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.504536    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xrh8x"
Jul 09 13:56:07.506555 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.506515    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 13:56:07.507356 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.507328    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-186.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 09 13:56:07.509494 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.509478    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 13:56:07.511434 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.511420    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 13:56:07.511490 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.511443    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 13:56:07.511490 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.511455    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasSufficientPID"
Jul 09 13:56:07.512134 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.512122    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 13:56:07.512134 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.512132    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 13:56:07.512208 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.512146    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 13:56:07.513258 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.513244    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 13:56:07.513390 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.513378    2579 policy_none.go:49] "None policy: Start"
Jul 09 13:56:07.513465 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.513392    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 13:56:07.513465 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.513405    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 13:56:07.513873 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.513858    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 13:56:07.513938 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.513882    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 13:56:07.513938 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.513895    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasSufficientPID"
Jul 09 13:56:07.514749 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.514723    2579 policy_none.go:47] "Start"
Jul 09 13:56:07.553521 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.553507    2579 manager.go:337] "Starting Device Plugin manager"
Jul 09 13:56:07.553585 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.553541    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 13:56:07.553585 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.553551    2579 server.go:87] "Starting device plugin registration server"
Jul 09 13:56:07.553776 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.553764    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 13:56:07.553856 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.553780    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 13:56:07.553910 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.553879    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 13:56:07.553961 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.553951    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 13:56:07.553961 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.553960    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 13:56:07.554440 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.554421    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 13:56:07.554512 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.554476    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:07.654334 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.654311    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 13:56:07.655497 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.655182    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 13:56:07.655497 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.655208    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 13:56:07.655497 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.655218    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasSufficientPID"
Jul 09 13:56:07.655497 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.655243    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-186.ec2.internal"
Jul 09 13:56:07.663200 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.663156    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-186.ec2.internal"
Jul 09 13:56:07.663200 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.663176    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-186.ec2.internal\": node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:07.681524 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.681503    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:07.703795 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.703773    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 13:56:07.703862 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.703814    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 13:56:07.703862 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.703836    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 13:56:07.703933 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.703865    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 13:56:07.706347 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.706327    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 13:56:07.782018 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.781987    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:07.804495 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.804462    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-186.ec2.internal"]
Jul 09 13:56:07.804564 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.804537    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 13:56:07.805443 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.805428    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 13:56:07.805518 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.805456    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 13:56:07.805518 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.805468    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasSufficientPID"
Jul 09 13:56:07.806845 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.806831    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 13:56:07.806963 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.806948    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal"
Jul 09 13:56:07.807032 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.807004    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 13:56:07.807597 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.807578    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 13:56:07.807681 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.807611    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 13:56:07.807681 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.807620    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 13:56:07.807681 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.807628    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasSufficientPID"
Jul 09 13:56:07.807681 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.807644    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 13:56:07.807681 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.807659    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasSufficientPID"
Jul 09 13:56:07.808947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.808931    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-186.ec2.internal"
Jul 09 13:56:07.809045 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.808958    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 13:56:07.809576 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.809555    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 13:56:07.809576 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.809578    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 13:56:07.809674 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.809588    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeHasSufficientPID"
Jul 09 13:56:07.836570 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.836553    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-186.ec2.internal\" not found" node="ip-10-0-128-186.ec2.internal"
Jul 09 13:56:07.840683 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.840669    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-186.ec2.internal\" not found" node="ip-10-0-128-186.ec2.internal"
Jul 09 13:56:07.882347 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.882332    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:07.896531 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.896514    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ee35923331e9eb54dbfecdcdc1ca917-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal\" (UID: \"7ee35923331e9eb54dbfecdcdc1ca917\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal"
Jul 09 13:56:07.896605 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.896539    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ee35923331e9eb54dbfecdcdc1ca917-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal\" (UID: \"7ee35923331e9eb54dbfecdcdc1ca917\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal"
Jul 09 13:56:07.896605 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.896555    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/92fbe84b040827f8e74fa00dccfc7dbd-config\") pod \"kube-apiserver-proxy-ip-10-0-128-186.ec2.internal\" (UID: \"92fbe84b040827f8e74fa00dccfc7dbd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-186.ec2.internal"
Jul 09 13:56:07.983434 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:07.983358    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:07.996706 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.996688    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ee35923331e9eb54dbfecdcdc1ca917-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal\" (UID: \"7ee35923331e9eb54dbfecdcdc1ca917\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal"
Jul 09 13:56:07.996778 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.996716    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ee35923331e9eb54dbfecdcdc1ca917-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal\" (UID: \"7ee35923331e9eb54dbfecdcdc1ca917\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal"
Jul 09 13:56:07.996778 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.996735    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/92fbe84b040827f8e74fa00dccfc7dbd-config\") pod \"kube-apiserver-proxy-ip-10-0-128-186.ec2.internal\" (UID: \"92fbe84b040827f8e74fa00dccfc7dbd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-186.ec2.internal"
Jul 09 13:56:07.996839 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.996803    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/92fbe84b040827f8e74fa00dccfc7dbd-config\") pod \"kube-apiserver-proxy-ip-10-0-128-186.ec2.internal\" (UID: \"92fbe84b040827f8e74fa00dccfc7dbd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-186.ec2.internal"
Jul 09 13:56:07.996839 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.996807    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ee35923331e9eb54dbfecdcdc1ca917-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal\" (UID: \"7ee35923331e9eb54dbfecdcdc1ca917\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal"
Jul 09 13:56:07.996839 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:07.996816    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ee35923331e9eb54dbfecdcdc1ca917-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal\" (UID: \"7ee35923331e9eb54dbfecdcdc1ca917\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal"
Jul 09 13:56:08.083873 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:08.083836    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:08.139404 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:08.139379    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal"
Jul 09 13:56:08.144545 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:08.144528    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-186.ec2.internal"
Jul 09 13:56:08.184010 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:08.183988    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:09.399248 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:09.399226    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:09.499806 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:09.499779    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:09.547345 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:09.547318    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 13:56:09.547934 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:09.547470    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 13:56:09.600429 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:09.600408    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:09.625324 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:09.625291    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 13:51:08 +0000 UTC" deadline="2027-12-31 18:34:01.481050339 +0000 UTC"
Jul 09 13:56:09.625324 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:09.625322    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12964h37m51.855731129s"
Jul 09 13:56:09.700554 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:09.700495    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:09.801090 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:09.801062    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:09.842322 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:09.842285    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ee35923331e9eb54dbfecdcdc1ca917.slice/crio-20a7db63f84797f5e2c283787c36fea33f940e42c0a9c3d07b3adb2b4cc12e86 WatchSource:0}: Error finding container 20a7db63f84797f5e2c283787c36fea33f940e42c0a9c3d07b3adb2b4cc12e86: Status 404 returned error can't find the container with id 20a7db63f84797f5e2c283787c36fea33f940e42c0a9c3d07b3adb2b4cc12e86
Jul 09 13:56:09.846443 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:09.846429    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 13:56:09.901839 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:09.901818    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:09.925278 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:09.925251    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92fbe84b040827f8e74fa00dccfc7dbd.slice/crio-15e8cec7d88c4ffe57e7345f38c63e06062915f3df12678b5645a29a3b912780 WatchSource:0}: Error finding container 15e8cec7d88c4ffe57e7345f38c63e06062915f3df12678b5645a29a3b912780: Status 404 returned error can't find the container with id 15e8cec7d88c4ffe57e7345f38c63e06062915f3df12678b5645a29a3b912780
Jul 09 13:56:10.002567 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:10.002507    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:10.103382 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:10.103355    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-186.ec2.internal\" not found"
Jul 09 13:56:10.139813 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.139777    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 13:56:10.179375 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.179351    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 13:56:10.208477 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.208457    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal"
Jul 09 13:56:10.211081 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.211068    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 13:56:10.220289 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.220270    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 13:56:10.221456 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.221443    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-186.ec2.internal"
Jul 09 13:56:10.232365 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.232353    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 13:56:10.588234 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.587986    2579 apiserver.go:52] "Watching apiserver"
Jul 09 13:56:10.594626 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.594603    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 13:56:10.597696 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.597673    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-sv4nr","openshift-dns/node-resolver-wr822","openshift-image-registry/node-ca-8rxdv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal","openshift-multus/multus-additional-cni-plugins-mpgcc","openshift-multus/network-metrics-daemon-jjd92","openshift-network-diagnostics/network-check-target-cp59j","openshift-ovn-kubernetes/ovnkube-node-z2w5j","kube-system/konnectivity-agent-jhxx7","kube-system/kube-apiserver-proxy-ip-10-0-128-186.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh","openshift-cluster-node-tuning-operator/tuned-6t6rv","openshift-multus/multus-xfhvs","openshift-network-operator/iptables-alerter-k5fnm"]
Jul 09 13:56:10.599964 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.599941    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.601178 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.601068    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wr822"
Jul 09 13:56:10.602417 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.602383    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8rxdv"
Jul 09 13:56:10.603308 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.602729    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 13:56:10.603308 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.602998    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 13:56:10.603308 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.603150    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 13:56:10.603308 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.603194    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 13:56:10.603462 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.603400    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 13:56:10.603608 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.603592    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 13:56:10.603861 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.603839    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kcnns\""
Jul 09 13:56:10.604143 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.604127    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:10.604222 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:10.604193    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:10.604797 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.604779    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 13:56:10.605130 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.605111    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 13:56:10.605323 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.605306    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kft7q\""
Jul 09 13:56:10.605509 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.605488    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-w2fbl\""
Jul 09 13:56:10.605876 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.605857    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 13:56:10.605961 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.605935    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.607665 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.607643    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 13:56:10.608248 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.608228    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hhq9q\""
Jul 09 13:56:10.608407 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.608385    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 13:56:10.608484 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.608441    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 13:56:10.609227 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.609132    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:10.609227 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:10.609189    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:10.611010 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.610990    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:10.611095 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:10.611056    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:10.612654 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.612614    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.617241    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.617431    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jhxx7"
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.617796    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wsnjp\""
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.617998    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.618090    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.618174    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.618340    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.618515    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.618716    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9cc463f4-0f20-4804-8e41-3c703171ae04-multus-daemon-config\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.618771    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wl2n\" (UniqueName: \"kubernetes.io/projected/fb705aa3-dcba-4a7f-ae12-32ccef61e542-kube-api-access-5wl2n\") pod \"network-metrics-daemon-jjd92\" (UID: \"fb705aa3-dcba-4a7f-ae12-32ccef61e542\") " pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.618803    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9cc463f4-0f20-4804-8e41-3c703171ae04-cni-binary-copy\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.618848    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-multus-socket-dir-parent\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.618878    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-etc-kubernetes\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.618907    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/950143f5-da10-4fa7-897f-cd547c9932d0-hosts-file\") pod \"node-resolver-wr822\" (UID: \"950143f5-da10-4fa7-897f-cd547c9932d0\") " pod="openshift-dns/node-resolver-wr822"
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.618935    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9039e54-a69f-4385-96ea-25d6ffa237c0-cnibin\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.618964    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9039e54-a69f-4385-96ea-25d6ffa237c0-os-release\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.618994    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-os-release\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619018    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-run-k8s-cni-cncf-io\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619049    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-var-lib-kubelet\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619077    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-hostroot\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.620614 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619107    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs\") pod \"network-metrics-daemon-jjd92\" (UID: \"fb705aa3-dcba-4a7f-ae12-32ccef61e542\") " pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619136    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-run-multus-certs\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619160    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/950143f5-da10-4fa7-897f-cd547c9932d0-tmp-dir\") pod \"node-resolver-wr822\" (UID: \"950143f5-da10-4fa7-897f-cd547c9932d0\") " pod="openshift-dns/node-resolver-wr822"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619232    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9039e54-a69f-4385-96ea-25d6ffa237c0-system-cni-dir\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619263    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9039e54-a69f-4385-96ea-25d6ffa237c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619430    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-multus-cni-dir\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619470    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-var-lib-cni-multus\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619497    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-multus-conf-dir\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619545    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln2rj\" (UniqueName: \"kubernetes.io/projected/9cc463f4-0f20-4804-8e41-3c703171ae04-kube-api-access-ln2rj\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619572    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b39fdbde-6632-4711-a42a-67a86d708c58-serviceca\") pod \"node-ca-8rxdv\" (UID: \"b39fdbde-6632-4711-a42a-67a86d708c58\") " pod="openshift-image-registry/node-ca-8rxdv"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619596    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb88988f-93b8-458f-8e2a-19c0ec6db469-kubelet-config\") pod \"global-pull-secret-syncer-sv4nr\" (UID: \"fb88988f-93b8-458f-8e2a-19c0ec6db469\") " pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619629    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9039e54-a69f-4385-96ea-25d6ffa237c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619704    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-system-cni-dir\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619773    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-cnibin\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619814    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb88988f-93b8-458f-8e2a-19c0ec6db469-dbus\") pod \"global-pull-secret-syncer-sv4nr\" (UID: \"fb88988f-93b8-458f-8e2a-19c0ec6db469\") " pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619846    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-var-lib-cni-bin\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619886    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lgnq\" (UniqueName: \"kubernetes.io/projected/950143f5-da10-4fa7-897f-cd547c9932d0-kube-api-access-4lgnq\") pod \"node-resolver-wr822\" (UID: \"950143f5-da10-4fa7-897f-cd547c9932d0\") " pod="openshift-dns/node-resolver-wr822"
Jul 09 13:56:10.621711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619915    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b39fdbde-6632-4711-a42a-67a86d708c58-host\") pod \"node-ca-8rxdv\" (UID: \"b39fdbde-6632-4711-a42a-67a86d708c58\") " pod="openshift-image-registry/node-ca-8rxdv"
Jul 09 13:56:10.622461 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619945    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9039e54-a69f-4385-96ea-25d6ffa237c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.622461 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.619972    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9039e54-a69f-4385-96ea-25d6ffa237c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.622461 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.620024    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-run-netns\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.622461 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.620053    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvlr7\" (UniqueName: \"kubernetes.io/projected/b39fdbde-6632-4711-a42a-67a86d708c58-kube-api-access-wvlr7\") pod \"node-ca-8rxdv\" (UID: \"b39fdbde-6632-4711-a42a-67a86d708c58\") " pod="openshift-image-registry/node-ca-8rxdv"
Jul 09 13:56:10.622461 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.620083    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret\") pod \"global-pull-secret-syncer-sv4nr\" (UID: \"fb88988f-93b8-458f-8e2a-19c0ec6db469\") " pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:10.622461 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.620116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6c2m\" (UniqueName: \"kubernetes.io/projected/e9039e54-a69f-4385-96ea-25d6ffa237c0-kube-api-access-d6c2m\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.622461 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.620148    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzn84\" (UniqueName: \"kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84\") pod \"network-check-target-cp59j\" (UID: \"6f731495-5079-4f7d-9d17-b43d73d8e5d4\") " pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:10.622461 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.620903    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 13:56:10.622461 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.622023    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cfmng\""
Jul 09 13:56:10.622461 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.622144    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 13:56:10.622958 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.622464    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k5fnm"
Jul 09 13:56:10.624314 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.624291    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.626424 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.625340    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-b5mht\""
Jul 09 13:56:10.626424 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.625531    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 13:56:10.626424 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.625601    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 13:56:10.626424 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.625800    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 13:56:10.626424 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.626027    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 13:56:10.627995 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.627444    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.629475 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.629256    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 13:56:10.629475 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.629305    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-668gb\""
Jul 09 13:56:10.630245 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.629848    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 13:56:10.630245 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.630087    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9m2cr\""
Jul 09 13:56:10.631339 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.630969    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 13:56:10.631339 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.631063    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 13:56:10.631339 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.631172    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 13:56:10.648767 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.648719    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-l5h5p"
Jul 09 13:56:10.657648 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.657627    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-l5h5p"
Jul 09 13:56:10.710111 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.710085    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 13:56:10.720536 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720510    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9cc463f4-0f20-4804-8e41-3c703171ae04-multus-daemon-config\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.720664 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720552    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-run-netns\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.720664 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720592    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-sysconfig\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.720803 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720692    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-tuned\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.720803 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720736    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-multus-socket-dir-parent\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.720803 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720776    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-etc-kubernetes\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.720803 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720799    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/950143f5-da10-4fa7-897f-cd547c9932d0-hosts-file\") pod \"node-resolver-wr822\" (UID: \"950143f5-da10-4fa7-897f-cd547c9932d0\") " pod="openshift-dns/node-resolver-wr822"
Jul 09 13:56:10.720999 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720824    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9039e54-a69f-4385-96ea-25d6ffa237c0-os-release\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.720999 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720849    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-systemd-units\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.720999 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720849    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-multus-socket-dir-parent\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.720999 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720871    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-os-release\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.720999 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720876    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-etc-kubernetes\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.720999 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720927    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9039e54-a69f-4385-96ea-25d6ffa237c0-os-release\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.720999 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720932    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-os-release\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.720999 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720932    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/950143f5-da10-4fa7-897f-cd547c9932d0-hosts-file\") pod \"node-resolver-wr822\" (UID: \"950143f5-da10-4fa7-897f-cd547c9932d0\") " pod="openshift-dns/node-resolver-wr822"
Jul 09 13:56:10.720999 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720961    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-run-k8s-cni-cncf-io\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.720999 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720986    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-var-lib-kubelet\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.720999 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.720989    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-run-k8s-cni-cncf-io\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721011    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-sys-fs\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721024    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-var-lib-kubelet\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721038    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v72g4\" (UniqueName: \"kubernetes.io/projected/c16d2120-1596-414d-aac6-e29bed586f15-kube-api-access-v72g4\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721077    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-run-systemd\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721099    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-ovnkube-script-lib\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721124    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-systemd\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721147    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9cc463f4-0f20-4804-8e41-3c703171ae04-multus-daemon-config\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-run-multus-certs\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/950143f5-da10-4fa7-897f-cd547c9932d0-tmp-dir\") pod \"node-resolver-wr822\" (UID: \"950143f5-da10-4fa7-897f-cd547c9932d0\") " pod="openshift-dns/node-resolver-wr822"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721220    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9039e54-a69f-4385-96ea-25d6ffa237c0-system-cni-dir\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721242    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-log-socket\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721252    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-run-multus-certs\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721265    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12db20ba-2e42-4777-9558-d4f6ce9587e9-konnectivity-ca\") pod \"konnectivity-agent-jhxx7\" (UID: \"12db20ba-2e42-4777-9558-d4f6ce9587e9\") " pod="kube-system/konnectivity-agent-jhxx7"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721302    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9039e54-a69f-4385-96ea-25d6ffa237c0-system-cni-dir\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721332    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25e34376-81cd-40d5-bad5-d3ee222a2a92-tmp\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721360    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-multus-conf-dir\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.721546 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721394    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b39fdbde-6632-4711-a42a-67a86d708c58-serviceca\") pod \"node-ca-8rxdv\" (UID: \"b39fdbde-6632-4711-a42a-67a86d708c58\") " pod="openshift-image-registry/node-ca-8rxdv"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721419    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-slash\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721443    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zptwx\" (UniqueName: \"kubernetes.io/projected/25e34376-81cd-40d5-bad5-d3ee222a2a92-kube-api-access-zptwx\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721463    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/950143f5-da10-4fa7-897f-cd547c9932d0-tmp-dir\") pod \"node-resolver-wr822\" (UID: \"950143f5-da10-4fa7-897f-cd547c9932d0\") " pod="openshift-dns/node-resolver-wr822"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721469    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb88988f-93b8-458f-8e2a-19c0ec6db469-dbus\") pod \"global-pull-secret-syncer-sv4nr\" (UID: \"fb88988f-93b8-458f-8e2a-19c0ec6db469\") " pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721506    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-socket-dir\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721526    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-multus-conf-dir\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721530    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-run-openvswitch\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721564    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-cni-bin\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721658    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-var-lib-cni-bin\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721697    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb88988f-93b8-458f-8e2a-19c0ec6db469-dbus\") pod \"global-pull-secret-syncer-sv4nr\" (UID: \"fb88988f-93b8-458f-8e2a-19c0ec6db469\") " pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721697    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721736    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-var-lib-cni-bin\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721794    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-run-netns\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721821    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b39fdbde-6632-4711-a42a-67a86d708c58-serviceca\") pod \"node-ca-8rxdv\" (UID: \"b39fdbde-6632-4711-a42a-67a86d708c58\") " pod="openshift-image-registry/node-ca-8rxdv"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721829    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvlr7\" (UniqueName: \"kubernetes.io/projected/b39fdbde-6632-4711-a42a-67a86d708c58-kube-api-access-wvlr7\") pod \"node-ca-8rxdv\" (UID: \"b39fdbde-6632-4711-a42a-67a86d708c58\") " pod="openshift-image-registry/node-ca-8rxdv"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721852    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret\") pod \"global-pull-secret-syncer-sv4nr\" (UID: \"fb88988f-93b8-458f-8e2a-19c0ec6db469\") " pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:10.722374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721882    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e-host-slash\") pod \"iptables-alerter-k5fnm\" (UID: \"8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e\") " pod="openshift-network-operator/iptables-alerter-k5fnm"
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721911    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-sysctl-d\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:10.721933    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721939    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wl2n\" (UniqueName: \"kubernetes.io/projected/fb705aa3-dcba-4a7f-ae12-32ccef61e542-kube-api-access-5wl2n\") pod \"network-metrics-daemon-jjd92\" (UID: \"fb705aa3-dcba-4a7f-ae12-32ccef61e542\") " pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.721965    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-etc-selinux\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:10.722011    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret podName:fb88988f-93b8-458f-8e2a-19c0ec6db469 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:11.221978587 +0000 UTC m=+3.004353516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret") pod "global-pull-secret-syncer-sv4nr" (UID: "fb88988f-93b8-458f-8e2a-19c0ec6db469") : object "kube-system"/"original-pull-secret" not registered
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722011    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-run-netns\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722028    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-etc-openvswitch\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722164    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-sys\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722192    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-host\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722219    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9cc463f4-0f20-4804-8e41-3c703171ae04-cni-binary-copy\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722245    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9039e54-a69f-4385-96ea-25d6ffa237c0-cnibin\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722288    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9039e54-a69f-4385-96ea-25d6ffa237c0-cnibin\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722431    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-device-dir\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722466    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-node-log\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722490    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-lib-modules\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722515    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-hostroot\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.723186 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722552    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs\") pod \"network-metrics-daemon-jjd92\" (UID: \"fb705aa3-dcba-4a7f-ae12-32ccef61e542\") " pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722577    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e-iptables-alerter-script\") pod \"iptables-alerter-k5fnm\" (UID: \"8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e\") " pod="openshift-network-operator/iptables-alerter-k5fnm"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722602    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ksln\" (UniqueName: \"kubernetes.io/projected/8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e-kube-api-access-5ksln\") pod \"iptables-alerter-k5fnm\" (UID: \"8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e\") " pod="openshift-network-operator/iptables-alerter-k5fnm"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722625    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-run-ovn\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722646    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-var-lib-kubelet\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722673    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9039e54-a69f-4385-96ea-25d6ffa237c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722714    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9cc463f4-0f20-4804-8e41-3c703171ae04-cni-binary-copy\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722726    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722770    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-ovnkube-config\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722794    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs7f2\" (UniqueName: \"kubernetes.io/projected/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-kube-api-access-gs7f2\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722817    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-kubernetes\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722844    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-multus-cni-dir\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722868    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-var-lib-cni-multus\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722874    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9039e54-a69f-4385-96ea-25d6ffa237c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722896    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln2rj\" (UniqueName: \"kubernetes.io/projected/9cc463f4-0f20-4804-8e41-3c703171ae04-kube-api-access-ln2rj\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722910    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-hostroot\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.722922    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb88988f-93b8-458f-8e2a-19c0ec6db469-kubelet-config\") pod \"global-pull-secret-syncer-sv4nr\" (UID: \"fb88988f-93b8-458f-8e2a-19c0ec6db469\") " pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:10.724047 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:10.723007    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:10.723047    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs podName:fb705aa3-dcba-4a7f-ae12-32ccef61e542 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:11.223032741 +0000 UTC m=+3.005407655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs") pod "network-metrics-daemon-jjd92" (UID: "fb705aa3-dcba-4a7f-ae12-32ccef61e542") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9039e54-a69f-4385-96ea-25d6ffa237c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723113    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-var-lib-openvswitch\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723146    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-env-overrides\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723174    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-multus-cni-dir\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723214    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-run\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723255    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-system-cni-dir\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723326    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb88988f-93b8-458f-8e2a-19c0ec6db469-kubelet-config\") pod \"global-pull-secret-syncer-sv4nr\" (UID: \"fb88988f-93b8-458f-8e2a-19c0ec6db469\") " pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723358    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-cnibin\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723378    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-host-var-lib-cni-multus\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723392    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-system-cni-dir\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723407    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-kubelet\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723451    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12db20ba-2e42-4777-9558-d4f6ce9587e9-agent-certs\") pod \"konnectivity-agent-jhxx7\" (UID: \"12db20ba-2e42-4777-9558-d4f6ce9587e9\") " pod="kube-system/konnectivity-agent-jhxx7"
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723483    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lgnq\" (UniqueName: \"kubernetes.io/projected/950143f5-da10-4fa7-897f-cd547c9932d0-kube-api-access-4lgnq\") pod \"node-resolver-wr822\" (UID: \"950143f5-da10-4fa7-897f-cd547c9932d0\") " pod="openshift-dns/node-resolver-wr822"
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723399    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9cc463f4-0f20-4804-8e41-3c703171ae04-cnibin\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723511    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b39fdbde-6632-4711-a42a-67a86d708c58-host\") pod \"node-ca-8rxdv\" (UID: \"b39fdbde-6632-4711-a42a-67a86d708c58\") " pod="openshift-image-registry/node-ca-8rxdv"
Jul 09 13:56:10.724573 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723615    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9039e54-a69f-4385-96ea-25d6ffa237c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.725429 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723653    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9039e54-a69f-4385-96ea-25d6ffa237c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.725429 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.723784    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b39fdbde-6632-4711-a42a-67a86d708c58-host\") pod \"node-ca-8rxdv\" (UID: \"b39fdbde-6632-4711-a42a-67a86d708c58\") " pod="openshift-image-registry/node-ca-8rxdv"
Jul 09 13:56:10.725429 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.724073    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9039e54-a69f-4385-96ea-25d6ffa237c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.725429 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.724101    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-run-ovn-kubernetes\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.725429 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.724124    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-cni-netd\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.725429 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.724162    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-ovn-node-metrics-cert\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.725429 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.724190    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6c2m\" (UniqueName: \"kubernetes.io/projected/e9039e54-a69f-4385-96ea-25d6ffa237c0-kube-api-access-d6c2m\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.725429 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.724254    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzn84\" (UniqueName: \"kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84\") pod \"network-check-target-cp59j\" (UID: \"6f731495-5079-4f7d-9d17-b43d73d8e5d4\") " pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:10.725429 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.724286    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-registration-dir\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.725429 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.724308    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-modprobe-d\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.725429 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.724329    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-sysctl-conf\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.725429 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.724549    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9039e54-a69f-4385-96ea-25d6ffa237c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.725429 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.724656    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9039e54-a69f-4385-96ea-25d6ffa237c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.732382 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.732355    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 13:56:10.733586 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:10.733313    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 13:56:10.733586 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:10.733336    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 13:56:10.733586 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:10.733349    2579 projected.go:196] Error preparing data for projected volume kube-api-access-qzn84 for pod openshift-network-diagnostics/network-check-target-cp59j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:56:10.733586 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:10.733406    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84 podName:6f731495-5079-4f7d-9d17-b43d73d8e5d4 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:11.233389067 +0000 UTC m=+3.015763993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qzn84" (UniqueName: "kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84") pod "network-check-target-cp59j" (UID: "6f731495-5079-4f7d-9d17-b43d73d8e5d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:56:10.736345 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.736319    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lgnq\" (UniqueName: \"kubernetes.io/projected/950143f5-da10-4fa7-897f-cd547c9932d0-kube-api-access-4lgnq\") pod \"node-resolver-wr822\" (UID: \"950143f5-da10-4fa7-897f-cd547c9932d0\") " pod="openshift-dns/node-resolver-wr822"
Jul 09 13:56:10.736829 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.736806    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6c2m\" (UniqueName: \"kubernetes.io/projected/e9039e54-a69f-4385-96ea-25d6ffa237c0-kube-api-access-d6c2m\") pod \"multus-additional-cni-plugins-mpgcc\" (UID: \"e9039e54-a69f-4385-96ea-25d6ffa237c0\") " pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.736931 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.736853    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvlr7\" (UniqueName: \"kubernetes.io/projected/b39fdbde-6632-4711-a42a-67a86d708c58-kube-api-access-wvlr7\") pod \"node-ca-8rxdv\" (UID: \"b39fdbde-6632-4711-a42a-67a86d708c58\") " pod="openshift-image-registry/node-ca-8rxdv"
Jul 09 13:56:10.737687 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.737665    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wl2n\" (UniqueName: \"kubernetes.io/projected/fb705aa3-dcba-4a7f-ae12-32ccef61e542-kube-api-access-5wl2n\") pod \"network-metrics-daemon-jjd92\" (UID: \"fb705aa3-dcba-4a7f-ae12-32ccef61e542\") " pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:10.738448 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.738427    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln2rj\" (UniqueName: \"kubernetes.io/projected/9cc463f4-0f20-4804-8e41-3c703171ae04-kube-api-access-ln2rj\") pod \"multus-xfhvs\" (UID: \"9cc463f4-0f20-4804-8e41-3c703171ae04\") " pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.822881 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.822826    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-186.ec2.internal" event={"ID":"92fbe84b040827f8e74fa00dccfc7dbd","Type":"ContainerStarted","Data":"15e8cec7d88c4ffe57e7345f38c63e06062915f3df12678b5645a29a3b912780"}
Jul 09 13:56:10.823898 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.823871    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal" event={"ID":"7ee35923331e9eb54dbfecdcdc1ca917","Type":"ContainerStarted","Data":"20a7db63f84797f5e2c283787c36fea33f940e42c0a9c3d07b3adb2b4cc12e86"}
Jul 09 13:56:10.825301 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825276    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-run-ovn-kubernetes\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.825414 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-cni-netd\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.825414 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825345    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-run-ovn-kubernetes\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.825414 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825350    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-ovn-node-metrics-cert\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.825529 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825424    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-cni-netd\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.825529 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825431    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-registration-dir\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.825529 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825459    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-modprobe-d\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.825529 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825473    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-sysctl-conf\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.825529 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825474    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-registration-dir\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.825529 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825492    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-run-netns\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.825529 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825508    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-sysconfig\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.825529 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825523    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-tuned\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825549    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-systemd-units\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825577    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-sys-fs\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825579    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-run-netns\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825587    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-modprobe-d\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825601    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v72g4\" (UniqueName: \"kubernetes.io/projected/c16d2120-1596-414d-aac6-e29bed586f15-kube-api-access-v72g4\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825608    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-sysconfig\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825636    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-systemd-units\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825642    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-sysctl-conf\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825644    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-sys-fs\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825668    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-run-systemd\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825700    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-ovnkube-script-lib\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825726    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-systemd\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825729    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-run-systemd\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825769    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-log-socket\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825793    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12db20ba-2e42-4777-9558-d4f6ce9587e9-konnectivity-ca\") pod \"konnectivity-agent-jhxx7\" (UID: \"12db20ba-2e42-4777-9558-d4f6ce9587e9\") " pod="kube-system/konnectivity-agent-jhxx7"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825798    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-log-socket\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825806    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-systemd\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.825947 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825817    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25e34376-81cd-40d5-bad5-d3ee222a2a92-tmp\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825842    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-slash\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825865    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zptwx\" (UniqueName: \"kubernetes.io/projected/25e34376-81cd-40d5-bad5-d3ee222a2a92-kube-api-access-zptwx\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825889    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-socket-dir\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825916    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-slash\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825933    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-run-openvswitch\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825960    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-cni-bin\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.825984    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826023    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e-host-slash\") pod \"iptables-alerter-k5fnm\" (UID: \"8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e\") " pod="openshift-network-operator/iptables-alerter-k5fnm"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826046    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-sysctl-d\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826070    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-etc-selinux\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826103    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-etc-openvswitch\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826111    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826125    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-sys\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826157    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-run-openvswitch\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826177    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-sys\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826184    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-host\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.826994 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826198    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-socket-dir\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826211    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-device-dir\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826218    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e-host-slash\") pod \"iptables-alerter-k5fnm\" (UID: \"8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e\") " pod="openshift-network-operator/iptables-alerter-k5fnm"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826234    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-node-log\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826250    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-cni-bin\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826271    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-node-log\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826305    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-etc-selinux\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826309    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-host\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826315    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-ovnkube-script-lib\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826337    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-device-dir\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826348    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-etc-openvswitch\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-lib-modules\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826339    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-sysctl-d\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826400    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e-iptables-alerter-script\") pod \"iptables-alerter-k5fnm\" (UID: \"8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e\") " pod="openshift-network-operator/iptables-alerter-k5fnm"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826440    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-lib-modules\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826454    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ksln\" (UniqueName: \"kubernetes.io/projected/8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e-kube-api-access-5ksln\") pod \"iptables-alerter-k5fnm\" (UID: \"8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e\") " pod="openshift-network-operator/iptables-alerter-k5fnm"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826482    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-run-ovn\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.827929 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826506    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-var-lib-kubelet\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826528    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826552    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-ovnkube-config\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826571    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-run-ovn\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826576    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7f2\" (UniqueName: \"kubernetes.io/projected/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-kube-api-access-gs7f2\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826608    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-kubernetes\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826639    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-var-lib-openvswitch\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826664    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-env-overrides\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826685    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-run\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-kubelet\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826731    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12db20ba-2e42-4777-9558-d4f6ce9587e9-agent-certs\") pod \"konnectivity-agent-jhxx7\" (UID: \"12db20ba-2e42-4777-9558-d4f6ce9587e9\") " pod="kube-system/konnectivity-agent-jhxx7"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826810    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-var-lib-kubelet\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826812    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e-iptables-alerter-script\") pod \"iptables-alerter-k5fnm\" (UID: \"8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e\") " pod="openshift-network-operator/iptables-alerter-k5fnm"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.826845    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c16d2120-1596-414d-aac6-e29bed586f15-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.827057    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.827264    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-ovnkube-config\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.827288    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-var-lib-openvswitch\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.827334    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-kubernetes\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.828917 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.827377    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/25e34376-81cd-40d5-bad5-d3ee222a2a92-run\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.829854 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.827470    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-host-kubelet\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.829854 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.827544    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12db20ba-2e42-4777-9558-d4f6ce9587e9-konnectivity-ca\") pod \"konnectivity-agent-jhxx7\" (UID: \"12db20ba-2e42-4777-9558-d4f6ce9587e9\") " pod="kube-system/konnectivity-agent-jhxx7"
Jul 09 13:56:10.829854 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.827710    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-env-overrides\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.829854 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.828861    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/25e34376-81cd-40d5-bad5-d3ee222a2a92-etc-tuned\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.829854 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.828899    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-ovn-node-metrics-cert\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.829854 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.829346    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12db20ba-2e42-4777-9558-d4f6ce9587e9-agent-certs\") pod \"konnectivity-agent-jhxx7\" (UID: \"12db20ba-2e42-4777-9558-d4f6ce9587e9\") " pod="kube-system/konnectivity-agent-jhxx7"
Jul 09 13:56:10.830387 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.830364    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25e34376-81cd-40d5-bad5-d3ee222a2a92-tmp\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.835660 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.835638    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs7f2\" (UniqueName: \"kubernetes.io/projected/9af8ec3e-1ee9-492b-82ed-cdb9b44b3404-kube-api-access-gs7f2\") pod \"ovnkube-node-z2w5j\" (UID: \"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.837413 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.837382    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zptwx\" (UniqueName: \"kubernetes.io/projected/25e34376-81cd-40d5-bad5-d3ee222a2a92-kube-api-access-zptwx\") pod \"tuned-6t6rv\" (UID: \"25e34376-81cd-40d5-bad5-d3ee222a2a92\") " pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:10.839197 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.839146    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v72g4\" (UniqueName: \"kubernetes.io/projected/c16d2120-1596-414d-aac6-e29bed586f15-kube-api-access-v72g4\") pod \"aws-ebs-csi-driver-node-8cdbh\" (UID: \"c16d2120-1596-414d-aac6-e29bed586f15\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.839197 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.839167    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ksln\" (UniqueName: \"kubernetes.io/projected/8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e-kube-api-access-5ksln\") pod \"iptables-alerter-k5fnm\" (UID: \"8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e\") " pod="openshift-network-operator/iptables-alerter-k5fnm"
Jul 09 13:56:10.914374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.914340    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xfhvs"
Jul 09 13:56:10.922838 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.922818    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wr822"
Jul 09 13:56:10.932772 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.932737    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8rxdv"
Jul 09 13:56:10.940543 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.940523    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mpgcc"
Jul 09 13:56:10.946493 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.946473    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:10.957276 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.957256    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jhxx7"
Jul 09 13:56:10.965971 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.965949    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k5fnm"
Jul 09 13:56:10.972959 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.972940    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh"
Jul 09 13:56:10.978814 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:10.978795    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6t6rv"
Jul 09 13:56:11.229821 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:11.229793    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret\") pod \"global-pull-secret-syncer-sv4nr\" (UID: \"fb88988f-93b8-458f-8e2a-19c0ec6db469\") " pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:11.229980 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:11.229836    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs\") pod \"network-metrics-daemon-jjd92\" (UID: \"fb705aa3-dcba-4a7f-ae12-32ccef61e542\") " pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:11.229980 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:11.229963    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 13:56:11.230094 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:11.229981    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:56:11.230094 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:11.230040    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret podName:fb88988f-93b8-458f-8e2a-19c0ec6db469 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:12.230020566 +0000 UTC m=+4.012395479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret") pod "global-pull-secret-syncer-sv4nr" (UID: "fb88988f-93b8-458f-8e2a-19c0ec6db469") : object "kube-system"/"original-pull-secret" not registered
Jul 09 13:56:11.230094 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:11.230075    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs podName:fb705aa3-dcba-4a7f-ae12-32ccef61e542 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:12.23005627 +0000 UTC m=+4.012431200 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs") pod "network-metrics-daemon-jjd92" (UID: "fb705aa3-dcba-4a7f-ae12-32ccef61e542") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:56:11.330536 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:11.330504    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzn84\" (UniqueName: \"kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84\") pod \"network-check-target-cp59j\" (UID: \"6f731495-5079-4f7d-9d17-b43d73d8e5d4\") " pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:11.330687 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:11.330646    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 13:56:11.330687 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:11.330662    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 13:56:11.330687 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:11.330674    2579 projected.go:196] Error preparing data for projected volume kube-api-access-qzn84 for pod openshift-network-diagnostics/network-check-target-cp59j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:56:11.330796 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:11.330728    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84 podName:6f731495-5079-4f7d-9d17-b43d73d8e5d4 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:12.330709838 +0000 UTC m=+4.113084766 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzn84" (UniqueName: "kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84") pod "network-check-target-cp59j" (UID: "6f731495-5079-4f7d-9d17-b43d73d8e5d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:56:11.660907 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:11.660863    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 13:51:10 +0000 UTC" deadline="2028-03-04 11:59:35.367014338 +0000 UTC"
Jul 09 13:56:11.660907 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:11.660903    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14494h3m23.706116127s"
Jul 09 13:56:11.819379 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:11.819356    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:11.819479 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:11.819460    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:11.825846 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:11.825659    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb39fdbde_6632_4711_a42a_67a86d708c58.slice/crio-3eb9088f0fb0104093d9897164fd822d750d8f2cdd301873969c8e1af5cdb9a4 WatchSource:0}: Error finding container 3eb9088f0fb0104093d9897164fd822d750d8f2cdd301873969c8e1af5cdb9a4: Status 404 returned error can't find the container with id 3eb9088f0fb0104093d9897164fd822d750d8f2cdd301873969c8e1af5cdb9a4
Jul 09 13:56:11.843212 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:11.843185    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cc463f4_0f20_4804_8e41_3c703171ae04.slice/crio-d95909fce7fa4a5f1e943add695058838622c845d56d82f4d1682fe6b04f98a6 WatchSource:0}: Error finding container d95909fce7fa4a5f1e943add695058838622c845d56d82f4d1682fe6b04f98a6: Status 404 returned error can't find the container with id d95909fce7fa4a5f1e943add695058838622c845d56d82f4d1682fe6b04f98a6
Jul 09 13:56:12.236871 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.236839    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret\") pod \"global-pull-secret-syncer-sv4nr\" (UID: \"fb88988f-93b8-458f-8e2a-19c0ec6db469\") " pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:12.237053 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.236887    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs\") pod \"network-metrics-daemon-jjd92\" (UID: \"fb705aa3-dcba-4a7f-ae12-32ccef61e542\") " pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:12.237053 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:12.237006    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:56:12.237159 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:12.237062    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs podName:fb705aa3-dcba-4a7f-ae12-32ccef61e542 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:14.237043356 +0000 UTC m=+6.019418285 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs") pod "network-metrics-daemon-jjd92" (UID: "fb705aa3-dcba-4a7f-ae12-32ccef61e542") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:56:12.237159 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:12.237122    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 13:56:12.237159 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:12.237152    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret podName:fb88988f-93b8-458f-8e2a-19c0ec6db469 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:14.237141717 +0000 UTC m=+6.019516632 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret") pod "global-pull-secret-syncer-sv4nr" (UID: "fb88988f-93b8-458f-8e2a-19c0ec6db469") : object "kube-system"/"original-pull-secret" not registered
Jul 09 13:56:12.338185 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.338109    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzn84\" (UniqueName: \"kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84\") pod \"network-check-target-cp59j\" (UID: \"6f731495-5079-4f7d-9d17-b43d73d8e5d4\") " pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:12.338341 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:12.338292    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 13:56:12.338341 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:12.338314    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 13:56:12.338341 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:12.338327    2579 projected.go:196] Error preparing data for projected volume kube-api-access-qzn84 for pod openshift-network-diagnostics/network-check-target-cp59j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:56:12.338504 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:12.338390    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84 podName:6f731495-5079-4f7d-9d17-b43d73d8e5d4 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:14.33837061 +0000 UTC m=+6.120745526 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzn84" (UniqueName: "kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84") pod "network-check-target-cp59j" (UID: "6f731495-5079-4f7d-9d17-b43d73d8e5d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:56:12.661374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.661313    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 13:51:10 +0000 UTC" deadline="2028-04-15 05:02:15.111638336 +0000 UTC"
Jul 09 13:56:12.661374 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.661373    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15495h6m2.45026987s"
Jul 09 13:56:12.820887 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.820861    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:12.821046 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:12.820982    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:12.822911 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.822875    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:12.823022 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:12.822991    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:12.834761 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.834715    2579 generic.go:356] "Generic (PLEG): container finished" podID="7ee35923331e9eb54dbfecdcdc1ca917" containerID="ffdb6661ce5d4e7794a02c5330d3889afcc940a65d0f3721bdb5d0570a0ad923" exitCode=0
Jul 09 13:56:12.835565 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.835511    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal" event={"ID":"7ee35923331e9eb54dbfecdcdc1ca917","Type":"ContainerDied","Data":"ffdb6661ce5d4e7794a02c5330d3889afcc940a65d0f3721bdb5d0570a0ad923"}
Jul 09 13:56:12.851807 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.851783    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh" event={"ID":"c16d2120-1596-414d-aac6-e29bed586f15","Type":"ContainerStarted","Data":"34a88c115c17a015ff1beb1f3f0d977051773d91092f386599d3d5f30bb13fda"}
Jul 09 13:56:12.859583 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.859486    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wr822" event={"ID":"950143f5-da10-4fa7-897f-cd547c9932d0","Type":"ContainerStarted","Data":"2104b7a0db90b106c3366ec6ecf01fce728bba69418beb8b3391879d2b365d82"}
Jul 09 13:56:12.877196 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.877167    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpgcc" event={"ID":"e9039e54-a69f-4385-96ea-25d6ffa237c0","Type":"ContainerStarted","Data":"e5b8cb96e18356edae52f3296d28079dd97994014c5a7c9384332a87534ee780"}
Jul 09 13:56:12.892213 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.892185    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j" event={"ID":"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404","Type":"ContainerStarted","Data":"fbf2ca229d26a187bfa06f7830d080219a3f3fca2a467b8575bf33cdf10dd145"}
Jul 09 13:56:12.901294 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.901270    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6t6rv" event={"ID":"25e34376-81cd-40d5-bad5-d3ee222a2a92","Type":"ContainerStarted","Data":"ee4cbabfc02ab4e6da8fdde5e14b677a08156aec5764fcb43b323c56554f4704"}
Jul 09 13:56:12.912199 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.912093    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xfhvs" event={"ID":"9cc463f4-0f20-4804-8e41-3c703171ae04","Type":"ContainerStarted","Data":"d95909fce7fa4a5f1e943add695058838622c845d56d82f4d1682fe6b04f98a6"}
Jul 09 13:56:12.916338 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.915973    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-186.ec2.internal" event={"ID":"92fbe84b040827f8e74fa00dccfc7dbd","Type":"ContainerStarted","Data":"51f1e15f79dd8e0ef510c6090c3e869949573015c1fac03a9c7876c628bcef5c"}
Jul 09 13:56:12.917561 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.917506    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jhxx7" event={"ID":"12db20ba-2e42-4777-9558-d4f6ce9587e9","Type":"ContainerStarted","Data":"4867d5a28122f025305ead102202963459ee927ad8ab4853a1702de5d92b9321"}
Jul 09 13:56:12.930324 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.930272    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-186.ec2.internal" podStartSLOduration=2.930255517 podStartE2EDuration="2.930255517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:56:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:12.929282654 +0000 UTC m=+4.711657589" watchObservedRunningTime="2026-07-09 13:56:12.930255517 +0000 UTC m=+4.712630454"
Jul 09 13:56:12.942459 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.942384    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k5fnm" event={"ID":"8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e","Type":"ContainerStarted","Data":"542175a0f170165a271f3a65cb6571ae7d8bdeef41a5fc272acbd8daaed1be90"}
Jul 09 13:56:12.945678 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:12.945608    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8rxdv" event={"ID":"b39fdbde-6632-4711-a42a-67a86d708c58","Type":"ContainerStarted","Data":"3eb9088f0fb0104093d9897164fd822d750d8f2cdd301873969c8e1af5cdb9a4"}
Jul 09 13:56:13.819304 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:13.819266    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:13.819864 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:13.819441    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:13.958250 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:13.956714    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal" event={"ID":"7ee35923331e9eb54dbfecdcdc1ca917","Type":"ContainerStarted","Data":"de4d9dcf3ad4484b7b9d9445268a1ca0fd62ed7550f5cdd0df5006afe1efd27e"}
Jul 09 13:56:14.251181 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:14.251140    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret\") pod \"global-pull-secret-syncer-sv4nr\" (UID: \"fb88988f-93b8-458f-8e2a-19c0ec6db469\") " pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:14.251370 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:14.251203    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs\") pod \"network-metrics-daemon-jjd92\" (UID: \"fb705aa3-dcba-4a7f-ae12-32ccef61e542\") " pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:14.251436 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:14.251369    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:56:14.251436 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:14.251433    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs podName:fb705aa3-dcba-4a7f-ae12-32ccef61e542 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:18.251412555 +0000 UTC m=+10.033787468 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs") pod "network-metrics-daemon-jjd92" (UID: "fb705aa3-dcba-4a7f-ae12-32ccef61e542") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:56:14.251877 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:14.251857    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 13:56:14.251955 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:14.251915    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret podName:fb88988f-93b8-458f-8e2a-19c0ec6db469 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:18.251898895 +0000 UTC m=+10.034273810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret") pod "global-pull-secret-syncer-sv4nr" (UID: "fb88988f-93b8-458f-8e2a-19c0ec6db469") : object "kube-system"/"original-pull-secret" not registered
Jul 09 13:56:14.352275 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:14.352235    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzn84\" (UniqueName: \"kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84\") pod \"network-check-target-cp59j\" (UID: \"6f731495-5079-4f7d-9d17-b43d73d8e5d4\") " pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:14.352428 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:14.352402    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 13:56:14.352428 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:14.352420    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 13:56:14.352496 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:14.352432    2579 projected.go:196] Error preparing data for projected volume kube-api-access-qzn84 for pod openshift-network-diagnostics/network-check-target-cp59j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:56:14.352496 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:14.352488    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84 podName:6f731495-5079-4f7d-9d17-b43d73d8e5d4 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:18.352470301 +0000 UTC m=+10.134845220 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzn84" (UniqueName: "kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84") pod "network-check-target-cp59j" (UID: "6f731495-5079-4f7d-9d17-b43d73d8e5d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:56:14.819900 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:14.819869    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:14.820371 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:14.820032    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:14.820371 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:14.820108    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:14.820371 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:14.820219    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:15.819532 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:15.819479    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:15.819707 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:15.819604    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:16.821311 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:16.821236    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:16.821726 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:16.821351    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:16.821726 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:16.821720    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:16.821879 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:16.821855    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:17.819823 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:17.819787    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:17.819987 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:17.819899    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:18.285986 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:18.285950    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret\") pod \"global-pull-secret-syncer-sv4nr\" (UID: \"fb88988f-93b8-458f-8e2a-19c0ec6db469\") " pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:18.286440 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:18.286007    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs\") pod \"network-metrics-daemon-jjd92\" (UID: \"fb705aa3-dcba-4a7f-ae12-32ccef61e542\") " pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:18.286440 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:18.286169    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:56:18.286440 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:18.286229    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs podName:fb705aa3-dcba-4a7f-ae12-32ccef61e542 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:26.286209827 +0000 UTC m=+18.068584763 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs") pod "network-metrics-daemon-jjd92" (UID: "fb705aa3-dcba-4a7f-ae12-32ccef61e542") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:56:18.286616 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:18.286546    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 13:56:18.286616 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:18.286581    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret podName:fb88988f-93b8-458f-8e2a-19c0ec6db469 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:26.286571017 +0000 UTC m=+18.068945931 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret") pod "global-pull-secret-syncer-sv4nr" (UID: "fb88988f-93b8-458f-8e2a-19c0ec6db469") : object "kube-system"/"original-pull-secret" not registered
Jul 09 13:56:18.386764 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:18.386711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzn84\" (UniqueName: \"kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84\") pod \"network-check-target-cp59j\" (UID: \"6f731495-5079-4f7d-9d17-b43d73d8e5d4\") " pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:18.386968 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:18.386942    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 13:56:18.386968 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:18.386973    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 13:56:18.387158 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:18.386990    2579 projected.go:196] Error preparing data for projected volume kube-api-access-qzn84 for pod openshift-network-diagnostics/network-check-target-cp59j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:56:18.387158 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:18.387047    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84 podName:6f731495-5079-4f7d-9d17-b43d73d8e5d4 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:26.387029028 +0000 UTC m=+18.169403946 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzn84" (UniqueName: "kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84") pod "network-check-target-cp59j" (UID: "6f731495-5079-4f7d-9d17-b43d73d8e5d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:56:18.820277 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:18.820239    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:18.820449 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:18.820369    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:18.820449 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:18.820427    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:18.820553 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:18.820529    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:19.820277 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:19.819808    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:19.820277 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:19.819936    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:20.819264 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:20.819203    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:20.819442 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:20.819327    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:20.819695 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:20.819680    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:20.822117 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:20.819802    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:21.819076 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:21.819040    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:21.819259 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:21.819166    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:22.822297 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:22.822265    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:22.822681 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:22.822349    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:22.822681 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:22.822364    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:22.822681 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:22.822449    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:23.819079 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:23.819036    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:23.819254 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:23.819160    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:24.821711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:24.821682    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:24.822171 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:24.821682    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:24.822171 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:24.821823    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:24.822171 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:24.821916    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:25.819844 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:25.819813    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:25.820029 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:25.819924    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:26.349603 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:26.349556    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs\") pod \"network-metrics-daemon-jjd92\" (UID: \"fb705aa3-dcba-4a7f-ae12-32ccef61e542\") " pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:26.350067 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:26.349651    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret\") pod \"global-pull-secret-syncer-sv4nr\" (UID: \"fb88988f-93b8-458f-8e2a-19c0ec6db469\") " pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:26.350067 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:26.349760    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:56:26.350067 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:26.349777    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 13:56:26.350067 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:26.349840    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs podName:fb705aa3-dcba-4a7f-ae12-32ccef61e542 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:42.349818957 +0000 UTC m=+34.132193883 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs") pod "network-metrics-daemon-jjd92" (UID: "fb705aa3-dcba-4a7f-ae12-32ccef61e542") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:56:26.350067 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:26.349862    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret podName:fb88988f-93b8-458f-8e2a-19c0ec6db469 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:42.349851893 +0000 UTC m=+34.132226840 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret") pod "global-pull-secret-syncer-sv4nr" (UID: "fb88988f-93b8-458f-8e2a-19c0ec6db469") : object "kube-system"/"original-pull-secret" not registered
Jul 09 13:56:26.450535 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:26.450503    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzn84\" (UniqueName: \"kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84\") pod \"network-check-target-cp59j\" (UID: \"6f731495-5079-4f7d-9d17-b43d73d8e5d4\") " pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:26.450740 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:26.450715    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 13:56:26.450740 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:26.450763    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 13:56:26.450867 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:26.450779    2579 projected.go:196] Error preparing data for projected volume kube-api-access-qzn84 for pod openshift-network-diagnostics/network-check-target-cp59j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:56:26.450867 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:26.450843    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84 podName:6f731495-5079-4f7d-9d17-b43d73d8e5d4 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:42.450825002 +0000 UTC m=+34.233199947 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzn84" (UniqueName: "kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84") pod "network-check-target-cp59j" (UID: "6f731495-5079-4f7d-9d17-b43d73d8e5d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:56:26.818944 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:26.818915    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:26.819111 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:26.819035    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:26.819111 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:26.819091    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:26.819262 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:26.819235    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:27.818931 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:27.818901    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:27.819370 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:27.819016    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:28.820009 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:28.819727    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:28.820009 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:28.819850    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:28.820009 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:28.819890    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:28.820009 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:28.819976    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:29.819238 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:29.819198    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:29.819425 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:29.819339    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:30.819734 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:30.819524    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:30.820513 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:30.819847    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:30.820513 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:30.819898    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:30.820513 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:30.819984    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:30.988544 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:30.988518    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8rxdv" event={"ID":"b39fdbde-6632-4711-a42a-67a86d708c58","Type":"ContainerStarted","Data":"c2f20e50fce6517bdd62a3dbdca6a9cca2fe0dbd854eb85e9f289f1a321ea2a4"}
Jul 09 13:56:30.990706 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:30.990682    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh" event={"ID":"c16d2120-1596-414d-aac6-e29bed586f15","Type":"ContainerStarted","Data":"1f7d951209f6662f1d31184a95e43816560f807052b105be00bf4769f8639e3e"}
Jul 09 13:56:30.992909 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:30.992492    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wr822" event={"ID":"950143f5-da10-4fa7-897f-cd547c9932d0","Type":"ContainerStarted","Data":"0feb0bf054341495e669254770dbd0846f02b358f89ca88ff2b378f0f91dbb81"}
Jul 09 13:56:30.994362 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:30.994333    2579 generic.go:356] "Generic (PLEG): container finished" podID="e9039e54-a69f-4385-96ea-25d6ffa237c0" containerID="0d58d5afb75c07682bff443767e110e1305fc4780907eb15b6c926f913d9e5d8" exitCode=0
Jul 09 13:56:30.994445 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:30.994411    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpgcc" event={"ID":"e9039e54-a69f-4385-96ea-25d6ffa237c0","Type":"ContainerDied","Data":"0d58d5afb75c07682bff443767e110e1305fc4780907eb15b6c926f913d9e5d8"}
Jul 09 13:56:30.996769 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:30.996736    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 13:56:30.997098 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:30.997077    2579 generic.go:356] "Generic (PLEG): container finished" podID="9af8ec3e-1ee9-492b-82ed-cdb9b44b3404" containerID="68713efccc09eef1b40fb860a52ed622e9e6231578a11b71b01ae844ac83122c" exitCode=1
Jul 09 13:56:30.997193 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:30.997152    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j" event={"ID":"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404","Type":"ContainerDied","Data":"68713efccc09eef1b40fb860a52ed622e9e6231578a11b71b01ae844ac83122c"}
Jul 09 13:56:30.999246 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:30.999222    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j" event={"ID":"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404","Type":"ContainerStarted","Data":"1d2ccfbd88d901798b972e709cbc5fff5b86a6c4359f362e95cac27881f89a76"}
Jul 09 13:56:30.999343 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:30.999309    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6t6rv" event={"ID":"25e34376-81cd-40d5-bad5-d3ee222a2a92","Type":"ContainerStarted","Data":"47d5e7cc5b8c3d1eec6d615fe0ba1ec46b17f4c5f1ac719cd9a3d8bd08c0dba9"}
Jul 09 13:56:31.000190 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:31.000168    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xfhvs" event={"ID":"9cc463f4-0f20-4804-8e41-3c703171ae04","Type":"ContainerStarted","Data":"f15f6a53ee6ba52a577a7bb394a8558ea837b3966afcd3378665a27743064554"}
Jul 09 13:56:31.002294 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:31.002267    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jhxx7" event={"ID":"12db20ba-2e42-4777-9558-d4f6ce9587e9","Type":"ContainerStarted","Data":"50f50c47a2222a331779aa26684e10897396f8284d6f7e12440c9ffa43ee34c5"}
Jul 09 13:56:31.006540 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:31.006491    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8rxdv" podStartSLOduration=12.635518174 podStartE2EDuration="23.006473914s" totalImagesPullingTime="10.37095574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:31.005631185 +0000 UTC m=+22.788006120" watchObservedRunningTime="2026-07-09 13:56:31.006473914 +0000 UTC m=+22.788848848"
Jul 09 13:56:31.006625 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:31.006599    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-186.ec2.internal" podStartSLOduration=21.006589829 podStartE2EDuration="21.006589829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:56:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:13.973917909 +0000 UTC m=+5.756292846" watchObservedRunningTime="2026-07-09 13:56:31.006589829 +0000 UTC m=+22.788964765"
Jul 09 13:56:31.024814 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:31.024617    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jhxx7" podStartSLOduration=12.670702419 podStartE2EDuration="23.024590701s" totalImagesPullingTime="10.353888282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:31.024406053 +0000 UTC m=+22.806780988" watchObservedRunningTime="2026-07-09 13:56:31.024590701 +0000 UTC m=+22.806965638"
Jul 09 13:56:31.049368 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:31.049331    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6t6rv" podStartSLOduration=4.3959742219999995 podStartE2EDuration="23.049318902s" totalImagesPullingTime="18.65334468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:31.04881552 +0000 UTC m=+22.831190456" watchObservedRunningTime="2026-07-09 13:56:31.049318902 +0000 UTC m=+22.831693837"
Jul 09 13:56:31.072886 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:31.072847    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xfhvs" podStartSLOduration=4.385185859 podStartE2EDuration="23.072835426s" totalImagesPullingTime="18.687649567s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:31.072659498 +0000 UTC m=+22.855034459" watchObservedRunningTime="2026-07-09 13:56:31.072835426 +0000 UTC m=+22.855210360"
Jul 09 13:56:31.115818 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:31.115777    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wr822" podStartSLOduration=4.472856442 podStartE2EDuration="23.115740012s" totalImagesPullingTime="18.64288357s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:31.11525632 +0000 UTC m=+22.897631254" watchObservedRunningTime="2026-07-09 13:56:31.115740012 +0000 UTC m=+22.898114946"
Jul 09 13:56:31.819460 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:31.819395    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:31.819629 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:31.819521    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:32.009047 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:32.009027    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 13:56:32.009999 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:32.009823    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j" event={"ID":"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404","Type":"ContainerStarted","Data":"71b3be76479281eaf0d8d2fdcf5b4153654a52f855efab8f2e9269c0b493ee8f"}
Jul 09 13:56:32.009999 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:32.009903    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j" event={"ID":"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404","Type":"ContainerStarted","Data":"dd6e7989004639d31341487e36e44a862a10c007e960dfc1e325e3d0633efd83"}
Jul 09 13:56:32.009999 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:32.009955    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j" event={"ID":"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404","Type":"ContainerStarted","Data":"62220e99a08e5f896a310b34ba8f3f0f918a19887a6e2df55b392679443ec239"}
Jul 09 13:56:32.009999 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:32.010001    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j" event={"ID":"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404","Type":"ContainerStarted","Data":"79cedb34678a4acc1fa1589aeb8e6e4a81a58cd1e9c2928cd4b8d7d347c4c9f3"}
Jul 09 13:56:32.055477 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:32.055450    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 13:56:32.683454 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:32.683317    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T13:56:32.05547229Z","UUID":"91a83771-9b60-4a6a-b209-37c457c3ced9","Handler":null,"Name":"","Endpoint":""}
Jul 09 13:56:32.685639 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:32.685235    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 13:56:32.685639 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:32.685278    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 13:56:32.823362 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:32.822932    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:32.823362 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:32.822969    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:32.823362 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:32.823067    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:32.823362 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:32.823182    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:33.013921 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:33.013837    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k5fnm" event={"ID":"8f9408d5-33ac-49fb-8ddb-5b4d76db0e3e","Type":"ContainerStarted","Data":"aa13d87f5e4fc5414c62fa2c29822d7bbc6cbd7342e56ff8fa285467390bc7c8"}
Jul 09 13:56:33.016403 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:33.016380    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh" event={"ID":"c16d2120-1596-414d-aac6-e29bed586f15","Type":"ContainerStarted","Data":"69faac406c4f429bdd18490b23c5eaadac346de73af995d3cbaf7ac2dba8180a"}
Jul 09 13:56:33.030001 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:33.029957    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-k5fnm" podStartSLOduration=6.391508235 podStartE2EDuration="25.029939618s" totalImagesPullingTime="18.638431383s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:33.02945601 +0000 UTC m=+24.811830939" watchObservedRunningTime="2026-07-09 13:56:33.029939618 +0000 UTC m=+24.812314554"
Jul 09 13:56:33.819863 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:33.819669    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:33.820052 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:33.819941    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:34.021024 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:34.020985    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh" event={"ID":"c16d2120-1596-414d-aac6-e29bed586f15","Type":"ContainerStarted","Data":"67039558943fe8e9c813e56a3ca2033301964ddff0d522c19d32c308d4716788"}
Jul 09 13:56:34.024485 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:34.024466    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 13:56:34.024910 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:34.024889    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j" event={"ID":"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404","Type":"ContainerStarted","Data":"1294f8bc7bf21fe94e18ba433868b7be7c88fc877e5bb0a23c85f5c46c74669b"}
Jul 09 13:56:34.819772 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:34.819724    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:34.819931 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:34.819724    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:34.819931 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:34.819901    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:34.820039 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:34.819942    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:35.522272 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:35.522117    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jhxx7"
Jul 09 13:56:35.522859 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:35.522743    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jhxx7"
Jul 09 13:56:35.542166 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:35.542118    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8cdbh" podStartSLOduration=6.714647702 podStartE2EDuration="27.542103256s" totalImagesPullingTime="20.827455554s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:56:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:34.038345376 +0000 UTC m=+25.820720313" watchObservedRunningTime="2026-07-09 13:56:35.542103256 +0000 UTC m=+27.324478192"
Jul 09 13:56:35.819387 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:35.819216    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:35.819498 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:35.819476    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:36.030488 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:36.030363    2579 generic.go:356] "Generic (PLEG): container finished" podID="e9039e54-a69f-4385-96ea-25d6ffa237c0" containerID="368dc30ab42a755d31739ba38b526ea9fa9edd83b9e501990856c49f04d0fea4" exitCode=0
Jul 09 13:56:36.030488 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:36.030453    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpgcc" event={"ID":"e9039e54-a69f-4385-96ea-25d6ffa237c0","Type":"ContainerDied","Data":"368dc30ab42a755d31739ba38b526ea9fa9edd83b9e501990856c49f04d0fea4"}
Jul 09 13:56:36.033840 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:36.033821    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 13:56:36.034246 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:36.034173    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j" event={"ID":"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404","Type":"ContainerStarted","Data":"09c0543c00ae5e0335c43f5d191e5ced1cb159661e2078e9e7a93755a6325b63"}
Jul 09 13:56:36.034472 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:36.034448    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jhxx7"
Jul 09 13:56:36.034562 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:36.034484    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:36.034562 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:36.034534    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:36.034685 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:36.034585    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:36.034685 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:36.034615    2579 scope.go:117] "RemoveContainer" containerID="68713efccc09eef1b40fb860a52ed622e9e6231578a11b71b01ae844ac83122c"
Jul 09 13:56:36.034970 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:36.034955    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jhxx7"
Jul 09 13:56:36.053335 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:36.053315    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:36.053421 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:36.053402    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:56:36.829771 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:36.820792    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:36.829771 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:36.820936    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:36.829771 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:36.821036    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:36.829771 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:36.821406    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:37.038877 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:37.038804    2579 generic.go:356] "Generic (PLEG): container finished" podID="e9039e54-a69f-4385-96ea-25d6ffa237c0" containerID="656ea19fcff79d073dcd01722a8b05f045de846428d64e38301fa549ba3627da" exitCode=0
Jul 09 13:56:37.039026 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:37.038878    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpgcc" event={"ID":"e9039e54-a69f-4385-96ea-25d6ffa237c0","Type":"ContainerDied","Data":"656ea19fcff79d073dcd01722a8b05f045de846428d64e38301fa549ba3627da"}
Jul 09 13:56:37.043163 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:37.043147    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 13:56:37.043574 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:37.043552    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j" event={"ID":"9af8ec3e-1ee9-492b-82ed-cdb9b44b3404","Type":"ContainerStarted","Data":"1eb2240ea289677939aa6dff630876cfa843645bff3fb7057692173641a4663f"}
Jul 09 13:56:37.087988 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:37.087948    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j" podStartSLOduration=10.367610992 podStartE2EDuration="29.087934435s" totalImagesPullingTime="18.720323443s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:37.086790442 +0000 UTC m=+28.869165378" watchObservedRunningTime="2026-07-09 13:56:37.087934435 +0000 UTC m=+28.870309369"
Jul 09 13:56:37.565355 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:37.565138    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sv4nr"]
Jul 09 13:56:37.565558 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:37.565486    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:37.565620 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:37.565585    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:37.569256 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:37.569158    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cp59j"]
Jul 09 13:56:37.569418 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:37.569318    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:37.569477 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:37.569410    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:37.569853 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:37.569825    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jjd92"]
Jul 09 13:56:37.569956 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:37.569933    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:37.570061 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:37.570040    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:38.048256 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:38.048174    2579 generic.go:356] "Generic (PLEG): container finished" podID="e9039e54-a69f-4385-96ea-25d6ffa237c0" containerID="fc785d8ad8e69a5e35234e66576291b9a65ef6ceb37b2da6ba0f31440ba98e91" exitCode=0
Jul 09 13:56:38.048797 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:38.048264    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpgcc" event={"ID":"e9039e54-a69f-4385-96ea-25d6ffa237c0","Type":"ContainerDied","Data":"fc785d8ad8e69a5e35234e66576291b9a65ef6ceb37b2da6ba0f31440ba98e91"}
Jul 09 13:56:38.820837 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:38.820809    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:38.820989 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:38.820927    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:38.821106 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:38.821081    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:38.821209 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:38.821187    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:38.821269 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:38.821238    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:38.821330 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:38.821311    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:40.819373 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:40.819338    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:40.819373 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:40.819358    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:40.820093 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:40.819338    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:40.820093 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:40.819459    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:40.820093 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:40.819576    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:40.820093 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:40.819655    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:42.375424 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:42.375390    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret\") pod \"global-pull-secret-syncer-sv4nr\" (UID: \"fb88988f-93b8-458f-8e2a-19c0ec6db469\") " pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:42.375424 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:42.375432    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs\") pod \"network-metrics-daemon-jjd92\" (UID: \"fb705aa3-dcba-4a7f-ae12-32ccef61e542\") " pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:42.375944 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:42.375551    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:56:42.375944 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:42.375551    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 13:56:42.375944 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:42.375613    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs podName:fb705aa3-dcba-4a7f-ae12-32ccef61e542 nodeName:}" failed. No retries permitted until 2026-07-09 13:57:14.375595695 +0000 UTC m=+66.157970622 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs") pod "network-metrics-daemon-jjd92" (UID: "fb705aa3-dcba-4a7f-ae12-32ccef61e542") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:56:42.375944 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:42.375630    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret podName:fb88988f-93b8-458f-8e2a-19c0ec6db469 nodeName:}" failed. No retries permitted until 2026-07-09 13:57:14.375622124 +0000 UTC m=+66.157997039 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret") pod "global-pull-secret-syncer-sv4nr" (UID: "fb88988f-93b8-458f-8e2a-19c0ec6db469") : object "kube-system"/"original-pull-secret" not registered
Jul 09 13:56:42.476056 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:42.476027    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzn84\" (UniqueName: \"kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84\") pod \"network-check-target-cp59j\" (UID: \"6f731495-5079-4f7d-9d17-b43d73d8e5d4\") " pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:42.476197 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:42.476179    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 13:56:42.476235 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:42.476197    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 13:56:42.476235 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:42.476207    2579 projected.go:196] Error preparing data for projected volume kube-api-access-qzn84 for pod openshift-network-diagnostics/network-check-target-cp59j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:56:42.476306 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:42.476253    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84 podName:6f731495-5079-4f7d-9d17-b43d73d8e5d4 nodeName:}" failed. No retries permitted until 2026-07-09 13:57:14.47624068 +0000 UTC m=+66.258615593 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzn84" (UniqueName: "kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84") pod "network-check-target-cp59j" (UID: "6f731495-5079-4f7d-9d17-b43d73d8e5d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:56:42.819595 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:42.819419    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:42.819765 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:42.819422    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:42.819765 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:42.819703    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4nr" podUID="fb88988f-93b8-458f-8e2a-19c0ec6db469"
Jul 09 13:56:42.819860 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:42.819423    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:42.819860 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:42.819845    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp59j" podUID="6f731495-5079-4f7d-9d17-b43d73d8e5d4"
Jul 09 13:56:42.819977 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:42.819956    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:56:43.063093 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.062995    2579 generic.go:356] "Generic (PLEG): container finished" podID="e9039e54-a69f-4385-96ea-25d6ffa237c0" containerID="d9716d94346f2b2e36fff86a8fe655965e5c883659db321d63955afb45371896" exitCode=0
Jul 09 13:56:43.063093 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.063035    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpgcc" event={"ID":"e9039e54-a69f-4385-96ea-25d6ffa237c0","Type":"ContainerDied","Data":"d9716d94346f2b2e36fff86a8fe655965e5c883659db321d63955afb45371896"}
Jul 09 13:56:43.553788 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.553764    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-186.ec2.internal" event="NodeReady"
Jul 09 13:56:43.554199 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.553875    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 13:56:43.612252 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.612225    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jnpml"]
Jul 09 13:56:43.625214 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.625192    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-456t7"]
Jul 09 13:56:43.625323 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.625312    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:43.627995 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.627978    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 13:56:43.628256 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.628244    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 13:56:43.628323 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.628257    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dkvmm\""
Jul 09 13:56:43.642008 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.641989    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jnpml"]
Jul 09 13:56:43.642100 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.642019    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-456t7"]
Jul 09 13:56:43.642145 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.642096    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-456t7"
Jul 09 13:56:43.644625 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.644605    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 13:56:43.644723 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.644661    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 13:56:43.644801 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.644783    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-r54c2\""
Jul 09 13:56:43.645494 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.645480    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 13:56:43.787905 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.787872    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-tmp-dir\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:43.788077 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.787909    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41620e9f-76f0-410a-8b47-d1a33dca6722-cert\") pod \"ingress-canary-456t7\" (UID: \"41620e9f-76f0-410a-8b47-d1a33dca6722\") " pod="openshift-ingress-canary/ingress-canary-456t7"
Jul 09 13:56:43.788077 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.787979    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjchb\" (UniqueName: \"kubernetes.io/projected/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-kube-api-access-tjchb\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:43.788077 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.788011    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsf6h\" (UniqueName: \"kubernetes.io/projected/41620e9f-76f0-410a-8b47-d1a33dca6722-kube-api-access-rsf6h\") pod \"ingress-canary-456t7\" (UID: \"41620e9f-76f0-410a-8b47-d1a33dca6722\") " pod="openshift-ingress-canary/ingress-canary-456t7"
Jul 09 13:56:43.788077 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.788031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-config-volume\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:43.788077 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.788053    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:43.889297 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.889229    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjchb\" (UniqueName: \"kubernetes.io/projected/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-kube-api-access-tjchb\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:43.889297 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.889265    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsf6h\" (UniqueName: \"kubernetes.io/projected/41620e9f-76f0-410a-8b47-d1a33dca6722-kube-api-access-rsf6h\") pod \"ingress-canary-456t7\" (UID: \"41620e9f-76f0-410a-8b47-d1a33dca6722\") " pod="openshift-ingress-canary/ingress-canary-456t7"
Jul 09 13:56:43.889297 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.889289    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-config-volume\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:43.889511 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.889314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:43.889511 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.889371    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-tmp-dir\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:43.889511 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.889398    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41620e9f-76f0-410a-8b47-d1a33dca6722-cert\") pod \"ingress-canary-456t7\" (UID: \"41620e9f-76f0-410a-8b47-d1a33dca6722\") " pod="openshift-ingress-canary/ingress-canary-456t7"
Jul 09 13:56:43.889511 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:43.889457    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 13:56:43.889672 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:43.889518    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls podName:7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:44.389499297 +0000 UTC m=+36.171874209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls") pod "dns-default-jnpml" (UID: "7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9") : secret "dns-default-metrics-tls" not found
Jul 09 13:56:43.889900 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.889781    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-tmp-dir\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:43.890095 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.890075    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-config-volume\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:43.893586 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.893570    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41620e9f-76f0-410a-8b47-d1a33dca6722-cert\") pod \"ingress-canary-456t7\" (UID: \"41620e9f-76f0-410a-8b47-d1a33dca6722\") " pod="openshift-ingress-canary/ingress-canary-456t7"
Jul 09 13:56:43.898556 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.898536    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsf6h\" (UniqueName: \"kubernetes.io/projected/41620e9f-76f0-410a-8b47-d1a33dca6722-kube-api-access-rsf6h\") pod \"ingress-canary-456t7\" (UID: \"41620e9f-76f0-410a-8b47-d1a33dca6722\") " pod="openshift-ingress-canary/ingress-canary-456t7"
Jul 09 13:56:43.898654 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.898579    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjchb\" (UniqueName: \"kubernetes.io/projected/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-kube-api-access-tjchb\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:43.949753 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:43.949730    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-456t7"
Jul 09 13:56:44.069851 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:44.069820    2579 generic.go:356] "Generic (PLEG): container finished" podID="e9039e54-a69f-4385-96ea-25d6ffa237c0" containerID="bca22eaadb2b309ff4af97b5b440c51fd262700ce85558e303d3928e01e68ac5" exitCode=0
Jul 09 13:56:44.070023 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:44.069875    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpgcc" event={"ID":"e9039e54-a69f-4385-96ea-25d6ffa237c0","Type":"ContainerDied","Data":"bca22eaadb2b309ff4af97b5b440c51fd262700ce85558e303d3928e01e68ac5"}
Jul 09 13:56:44.136892 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:44.136820    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-456t7"]
Jul 09 13:56:44.141468 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:56:44.141409    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41620e9f_76f0_410a_8b47_d1a33dca6722.slice/crio-02a3f5ba82fbccca76171186964b5f9354ad239c335bc822881c7cae4cb18960 WatchSource:0}: Error finding container 02a3f5ba82fbccca76171186964b5f9354ad239c335bc822881c7cae4cb18960: Status 404 returned error can't find the container with id 02a3f5ba82fbccca76171186964b5f9354ad239c335bc822881c7cae4cb18960
Jul 09 13:56:44.393199 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:44.392947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:44.393356 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:44.393098    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 13:56:44.393356 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:44.393271    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls podName:7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:45.393253792 +0000 UTC m=+37.175628708 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls") pod "dns-default-jnpml" (UID: "7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9") : secret "dns-default-metrics-tls" not found
Jul 09 13:56:44.819015 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:44.818982    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:56:44.819737 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:44.818983    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:56:44.819737 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:44.819172    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:56:44.821684 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:44.821664    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 13:56:44.822807 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:44.822783    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 13:56:44.822807 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:44.822801    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qkthd\""
Jul 09 13:56:44.822985 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:44.822802    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mkkhr\""
Jul 09 13:56:44.822985 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:44.822907    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 13:56:44.823089 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:44.822787    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 13:56:45.075780 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:45.075676    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpgcc" event={"ID":"e9039e54-a69f-4385-96ea-25d6ffa237c0","Type":"ContainerStarted","Data":"37d8bb9913ca7fdb859d5f209f856c73d584143643b6071371cbbc2db4e5deb1"}
Jul 09 13:56:45.079813 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:45.077155    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-456t7" event={"ID":"41620e9f-76f0-410a-8b47-d1a33dca6722","Type":"ContainerStarted","Data":"02a3f5ba82fbccca76171186964b5f9354ad239c335bc822881c7cae4cb18960"}
Jul 09 13:56:45.106026 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:45.105971    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mpgcc" podStartSLOduration=7.89068329 podStartE2EDuration="37.105948053s" totalImagesPullingTime="29.215264763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:56:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:45.104269914 +0000 UTC m=+36.886644881" watchObservedRunningTime="2026-07-09 13:56:45.105948053 +0000 UTC m=+36.888322988"
Jul 09 13:56:45.400143 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:45.400107    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:45.400302 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:45.400246    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 13:56:45.400355 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:45.400325    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls podName:7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:47.400302481 +0000 UTC m=+39.182677414 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls") pod "dns-default-jnpml" (UID: "7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9") : secret "dns-default-metrics-tls" not found
Jul 09 13:56:47.083002 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:47.082965    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-456t7" event={"ID":"41620e9f-76f0-410a-8b47-d1a33dca6722","Type":"ContainerStarted","Data":"ccce963683013d3cc515ae91fd6641350dd9209a309d013dab23f27b23402972"}
Jul 09 13:56:47.103322 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:47.103276    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-456t7" podStartSLOduration=1.728494531 podStartE2EDuration="4.103261799s" totalImagesPullingTime="2.374767268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:47.101818813 +0000 UTC m=+38.884193748" watchObservedRunningTime="2026-07-09 13:56:47.103261799 +0000 UTC m=+38.885636734"
Jul 09 13:56:47.419223 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:47.419193    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:47.419392 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:47.419371    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 13:56:47.419459 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:47.419449    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls podName:7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:51.419429659 +0000 UTC m=+43.201804587 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls") pod "dns-default-jnpml" (UID: "7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9") : secret "dns-default-metrics-tls" not found
Jul 09 13:56:51.446198 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:51.446160    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:51.446686 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:51.446305    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 13:56:51.446686 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:51.446372    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls podName:7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:59.446355353 +0000 UTC m=+51.228730266 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls") pod "dns-default-jnpml" (UID: "7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9") : secret "dns-default-metrics-tls" not found
Jul 09 13:56:59.500786 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:56:59.500732    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:56:59.501140 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:59.500891    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 13:56:59.501140 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:56:59.500960    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls podName:7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9 nodeName:}" failed. No retries permitted until 2026-07-09 13:57:15.500945604 +0000 UTC m=+67.283320517 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls") pod "dns-default-jnpml" (UID: "7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9") : secret "dns-default-metrics-tls" not found
Jul 09 13:57:00.936281 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:00.936249    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"]
Jul 09 13:57:00.956454 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:00.956428    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"]
Jul 09 13:57:00.956581 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:00.956519    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"
Jul 09 13:57:00.960470 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:00.960439    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 09 13:57:00.960470 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:00.960441    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 09 13:57:00.960703 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:00.960464    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 09 13:57:00.960703 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:00.960492    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 09 13:57:00.999433 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:00.999410    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"]
Jul 09 13:57:01.010035 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.010016    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8039ae6d-e6a1-49d8-a8f8-771681558f0d-klusterlet-config\") pod \"klusterlet-addon-workmgr-589b67687c-9lcgd\" (UID: \"8039ae6d-e6a1-49d8-a8f8-771681558f0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"
Jul 09 13:57:01.010157 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.010046    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsp44\" (UniqueName: \"kubernetes.io/projected/8039ae6d-e6a1-49d8-a8f8-771681558f0d-kube-api-access-lsp44\") pod \"klusterlet-addon-workmgr-589b67687c-9lcgd\" (UID: \"8039ae6d-e6a1-49d8-a8f8-771681558f0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"
Jul 09 13:57:01.010157 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.010146    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8039ae6d-e6a1-49d8-a8f8-771681558f0d-tmp\") pod \"klusterlet-addon-workmgr-589b67687c-9lcgd\" (UID: \"8039ae6d-e6a1-49d8-a8f8-771681558f0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"
Jul 09 13:57:01.014544 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.014529    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.015938 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.015918    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"]
Jul 09 13:57:01.017278 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.017254    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 09 13:57:01.017407 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.017276    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 09 13:57:01.017534 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.017518    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 09 13:57:01.017610 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.017550    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 09 13:57:01.111103 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.111079    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.111220 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.111109    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8039ae6d-e6a1-49d8-a8f8-771681558f0d-tmp\") pod \"klusterlet-addon-workmgr-589b67687c-9lcgd\" (UID: \"8039ae6d-e6a1-49d8-a8f8-771681558f0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"
Jul 09 13:57:01.111220 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.111134    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8039ae6d-e6a1-49d8-a8f8-771681558f0d-klusterlet-config\") pod \"klusterlet-addon-workmgr-589b67687c-9lcgd\" (UID: \"8039ae6d-e6a1-49d8-a8f8-771681558f0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"
Jul 09 13:57:01.111220 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.111151    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsp44\" (UniqueName: \"kubernetes.io/projected/8039ae6d-e6a1-49d8-a8f8-771681558f0d-kube-api-access-lsp44\") pod \"klusterlet-addon-workmgr-589b67687c-9lcgd\" (UID: \"8039ae6d-e6a1-49d8-a8f8-771681558f0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"
Jul 09 13:57:01.111220 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.111176    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-hub\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.111220 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.111190    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbccw\" (UniqueName: \"kubernetes.io/projected/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-kube-api-access-qbccw\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.111220 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.111217    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.111444 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.111242    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.111444 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.111263    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-ca\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.111505 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.111466    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8039ae6d-e6a1-49d8-a8f8-771681558f0d-tmp\") pod \"klusterlet-addon-workmgr-589b67687c-9lcgd\" (UID: \"8039ae6d-e6a1-49d8-a8f8-771681558f0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"
Jul 09 13:57:01.114668 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.114652    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8039ae6d-e6a1-49d8-a8f8-771681558f0d-klusterlet-config\") pod \"klusterlet-addon-workmgr-589b67687c-9lcgd\" (UID: \"8039ae6d-e6a1-49d8-a8f8-771681558f0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"
Jul 09 13:57:01.123324 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.123304    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsp44\" (UniqueName: \"kubernetes.io/projected/8039ae6d-e6a1-49d8-a8f8-771681558f0d-kube-api-access-lsp44\") pod \"klusterlet-addon-workmgr-589b67687c-9lcgd\" (UID: \"8039ae6d-e6a1-49d8-a8f8-771681558f0d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"
Jul 09 13:57:01.211997 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.211934    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.211997 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.211976    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.212148 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.211995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-ca\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.212148 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.212034    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.212148 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.212086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-hub\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.212148 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.212105    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbccw\" (UniqueName: \"kubernetes.io/projected/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-kube-api-access-qbccw\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.230069 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.229903    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.231661 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.231636    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.231737 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.231688    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-ca\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.231737 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.231708    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-hub\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.231845 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.231830    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.249933 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.249907    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbccw\" (UniqueName: \"kubernetes.io/projected/fcd91fdb-9ec8-4f4b-828a-95c18bfb087c-kube-api-access-qbccw\") pod \"cluster-proxy-proxy-agent-976c5fb47-cxmkb\" (UID: \"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.265811 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.265786    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"
Jul 09 13:57:01.337234 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.337205    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 13:57:01.444826 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.444794    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"]
Jul 09 13:57:01.447596 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:57:01.447568    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8039ae6d_e6a1_49d8_a8f8_771681558f0d.slice/crio-777bb4e4bc4ba486520fbfa88cc4eb73fce8eec4b22f71f8ffb8d522370a73bb WatchSource:0}: Error finding container 777bb4e4bc4ba486520fbfa88cc4eb73fce8eec4b22f71f8ffb8d522370a73bb: Status 404 returned error can't find the container with id 777bb4e4bc4ba486520fbfa88cc4eb73fce8eec4b22f71f8ffb8d522370a73bb
Jul 09 13:57:01.490942 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:01.490871    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"]
Jul 09 13:57:01.494056 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:57:01.494031    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfcd91fdb_9ec8_4f4b_828a_95c18bfb087c.slice/crio-388e0e0009161e3bf0fb400cf827a6918338af18e3af6af143f777556fff5700 WatchSource:0}: Error finding container 388e0e0009161e3bf0fb400cf827a6918338af18e3af6af143f777556fff5700: Status 404 returned error can't find the container with id 388e0e0009161e3bf0fb400cf827a6918338af18e3af6af143f777556fff5700
Jul 09 13:57:02.120943 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:02.120908    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb" event={"ID":"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c","Type":"ContainerStarted","Data":"388e0e0009161e3bf0fb400cf827a6918338af18e3af6af143f777556fff5700"}
Jul 09 13:57:02.122278 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:02.122246    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd" event={"ID":"8039ae6d-e6a1-49d8-a8f8-771681558f0d","Type":"ContainerStarted","Data":"777bb4e4bc4ba486520fbfa88cc4eb73fce8eec4b22f71f8ffb8d522370a73bb"}
Jul 09 13:57:07.137919 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:07.137818    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb" event={"ID":"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c","Type":"ContainerStarted","Data":"3032c3b06f3af611a7fe5d77ad960629d5fa3f569c1607ea8949be4757c9ced8"}
Jul 09 13:57:07.137919 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:07.137881    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb" event={"ID":"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c","Type":"ContainerStarted","Data":"8479539d5f9b392941d880415c93ed37778e11a412dc9a1d8b1006591ee3eb81"}
Jul 09 13:57:07.137919 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:07.137897    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb" event={"ID":"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c","Type":"ContainerStarted","Data":"014641599d0ebc6765256256ba58a2785e40432944df7ecb2de1ee600bf03117"}
Jul 09 13:57:07.139152 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:07.139131    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd" event={"ID":"8039ae6d-e6a1-49d8-a8f8-771681558f0d","Type":"ContainerStarted","Data":"a73bc749ede0bd9b417e2342ed451cdc0caabd7ad14f373e00030c0c8a8c9a24"}
Jul 09 13:57:07.139245 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:07.139174    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"
Jul 09 13:57:07.140851 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:07.140832    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"
Jul 09 13:57:07.157922 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:07.157889    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb" podStartSLOduration=1.902518346 podStartE2EDuration="7.157876809s" totalImagesPullingTime="5.255358463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:57:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:57:07.157220548 +0000 UTC m=+58.939595482" watchObservedRunningTime="2026-07-09 13:57:07.157876809 +0000 UTC m=+58.940251744"
Jul 09 13:57:07.174788 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:07.174729    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd" podStartSLOduration=1.874836014 podStartE2EDuration="7.174716081s" totalImagesPullingTime="5.299880067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:57:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:57:07.174413135 +0000 UTC m=+58.956788081" watchObservedRunningTime="2026-07-09 13:57:07.174716081 +0000 UTC m=+58.957091014"
Jul 09 13:57:08.059068 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:08.059028    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2w5j"
Jul 09 13:57:14.408864 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:14.408823    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret\") pod \"global-pull-secret-syncer-sv4nr\" (UID: \"fb88988f-93b8-458f-8e2a-19c0ec6db469\") " pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:57:14.409289 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:14.408875    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs\") pod \"network-metrics-daemon-jjd92\" (UID: \"fb705aa3-dcba-4a7f-ae12-32ccef61e542\") " pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:57:14.411691 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:14.411673    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 13:57:14.411741 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:14.411674    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 13:57:14.419703 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:57:14.419687    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 09 13:57:14.419779 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:57:14.419741    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs podName:fb705aa3-dcba-4a7f-ae12-32ccef61e542 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:18.419727048 +0000 UTC m=+130.202101961 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs") pod "network-metrics-daemon-jjd92" (UID: "fb705aa3-dcba-4a7f-ae12-32ccef61e542") : secret "metrics-daemon-secret" not found
Jul 09 13:57:14.421971 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:14.421950    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb88988f-93b8-458f-8e2a-19c0ec6db469-original-pull-secret\") pod \"global-pull-secret-syncer-sv4nr\" (UID: \"fb88988f-93b8-458f-8e2a-19c0ec6db469\") " pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:57:14.509437 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:14.509400    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzn84\" (UniqueName: \"kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84\") pod \"network-check-target-cp59j\" (UID: \"6f731495-5079-4f7d-9d17-b43d73d8e5d4\") " pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:57:14.512452 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:14.512434    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 13:57:14.522499 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:14.522478    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 13:57:14.532925 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:14.532906    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzn84\" (UniqueName: \"kubernetes.io/projected/6f731495-5079-4f7d-9d17-b43d73d8e5d4-kube-api-access-qzn84\") pod \"network-check-target-cp59j\" (UID: \"6f731495-5079-4f7d-9d17-b43d73d8e5d4\") " pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:57:14.546407 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:14.546388    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mkkhr\""
Jul 09 13:57:14.550570 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:14.550558    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4nr"
Jul 09 13:57:14.554787 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:14.554771    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:57:14.713256 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:14.713228    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sv4nr"]
Jul 09 13:57:14.717333 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:57:14.717307    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb88988f_93b8_458f_8e2a_19c0ec6db469.slice/crio-82c7d994871d9f2f1667dffdf0240621849e2a542740c5383c9c356a9ad31ab1 WatchSource:0}: Error finding container 82c7d994871d9f2f1667dffdf0240621849e2a542740c5383c9c356a9ad31ab1: Status 404 returned error can't find the container with id 82c7d994871d9f2f1667dffdf0240621849e2a542740c5383c9c356a9ad31ab1
Jul 09 13:57:14.726607 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:14.726549    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cp59j"]
Jul 09 13:57:14.730249 ip-10-0-128-186 kubenswrapper[2579]: W0709 13:57:14.730225    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f731495_5079_4f7d_9d17_b43d73d8e5d4.slice/crio-aeb9b2e78b88725f9874fa74e53a720a0cf5880cdfa6b9f6f3bb347222fb4780 WatchSource:0}: Error finding container aeb9b2e78b88725f9874fa74e53a720a0cf5880cdfa6b9f6f3bb347222fb4780: Status 404 returned error can't find the container with id aeb9b2e78b88725f9874fa74e53a720a0cf5880cdfa6b9f6f3bb347222fb4780
Jul 09 13:57:15.158824 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:15.158793    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cp59j" event={"ID":"6f731495-5079-4f7d-9d17-b43d73d8e5d4","Type":"ContainerStarted","Data":"aeb9b2e78b88725f9874fa74e53a720a0cf5880cdfa6b9f6f3bb347222fb4780"}
Jul 09 13:57:15.159774 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:15.159738    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sv4nr" event={"ID":"fb88988f-93b8-458f-8e2a-19c0ec6db469","Type":"ContainerStarted","Data":"82c7d994871d9f2f1667dffdf0240621849e2a542740c5383c9c356a9ad31ab1"}
Jul 09 13:57:15.515955 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:15.515878    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:57:15.516340 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:57:15.516028    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 13:57:15.516340 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:57:15.516094    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls podName:7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9 nodeName:}" failed. No retries permitted until 2026-07-09 13:57:47.516073162 +0000 UTC m=+99.298448095 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls") pod "dns-default-jnpml" (UID: "7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9") : secret "dns-default-metrics-tls" not found
Jul 09 13:57:20.178232 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:20.178183    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cp59j" event={"ID":"6f731495-5079-4f7d-9d17-b43d73d8e5d4","Type":"ContainerStarted","Data":"7effea5d19dea14e0ebb06e47efc977e24d8db01ead7c19ce4ef9c50a1315df5"}
Jul 09 13:57:20.178680 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:20.178340    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:57:20.179520 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:20.179494    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sv4nr" event={"ID":"fb88988f-93b8-458f-8e2a-19c0ec6db469","Type":"ContainerStarted","Data":"84e0a717f060100d9eb84817874a3dfa2ea74581d631ae0e6b49a7c1edc5bfe3"}
Jul 09 13:57:20.195121 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:20.195080    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cp59j" podStartSLOduration=67.774358502 podStartE2EDuration="1m12.195066852s" totalImagesPullingTime="4.42070835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:57:20.193407279 +0000 UTC m=+71.975782215" watchObservedRunningTime="2026-07-09 13:57:20.195066852 +0000 UTC m=+71.977441785"
Jul 09 13:57:20.210340 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:20.210305    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sv4nr" podStartSLOduration=67.773182074 podStartE2EDuration="1m12.210292536s" totalImagesPullingTime="4.437110462s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:57:20.209798121 +0000 UTC m=+71.992173056" watchObservedRunningTime="2026-07-09 13:57:20.210292536 +0000 UTC m=+71.992667470"
Jul 09 13:57:47.543988 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:47.543946    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:57:47.544475 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:57:47.544071    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 13:57:47.544475 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:57:47.544153    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls podName:7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:51.5441333 +0000 UTC m=+163.326508229 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls") pod "dns-default-jnpml" (UID: "7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9") : secret "dns-default-metrics-tls" not found
Jul 09 13:57:51.188012 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:57:51.187981    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cp59j"
Jul 09 13:58:18.464479 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:58:18.464426    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs\") pod \"network-metrics-daemon-jjd92\" (UID: \"fb705aa3-dcba-4a7f-ae12-32ccef61e542\") " pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:58:18.464981 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:58:18.464581    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 09 13:58:18.464981 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:58:18.464654    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs podName:fb705aa3-dcba-4a7f-ae12-32ccef61e542 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:20.464639147 +0000 UTC m=+252.247014061 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs") pod "network-metrics-daemon-jjd92" (UID: "fb705aa3-dcba-4a7f-ae12-32ccef61e542") : secret "metrics-daemon-secret" not found
Jul 09 13:58:46.634346 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:58:46.634301    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jnpml" podUID="7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9"
Jul 09 13:58:47.433186 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:58:47.433156    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jnpml"
Jul 09 13:58:47.830928 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:58:47.830835    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-jjd92" podUID="fb705aa3-dcba-4a7f-ae12-32ccef61e542"
Jul 09 13:58:51.589124 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:58:51.589081    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 13:58:51.589501 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:58:51.589236    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 13:58:51.589501 ip-10-0-128-186 kubenswrapper[2579]: E0709 13:58:51.589295    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls podName:7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:53.589279594 +0000 UTC m=+285.371654507 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls") pod "dns-default-jnpml" (UID: "7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9") : secret "dns-default-metrics-tls" not found
Jul 09 13:59:02.819251 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:59:02.819178    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 13:59:07.140015 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:59:07.139921    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd" podUID="8039ae6d-e6a1-49d8-a8f8-771681558f0d" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/readyz\": dial tcp 10.134.0.8:8000: connect: connection refused"
Jul 09 13:59:07.491818 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:59:07.491788    2579 generic.go:356] "Generic (PLEG): container finished" podID="8039ae6d-e6a1-49d8-a8f8-771681558f0d" containerID="a73bc749ede0bd9b417e2342ed451cdc0caabd7ad14f373e00030c0c8a8c9a24" exitCode=1
Jul 09 13:59:07.491983 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:59:07.491827    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd" event={"ID":"8039ae6d-e6a1-49d8-a8f8-771681558f0d","Type":"ContainerDied","Data":"a73bc749ede0bd9b417e2342ed451cdc0caabd7ad14f373e00030c0c8a8c9a24"}
Jul 09 13:59:07.492122 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:59:07.492110    2579 scope.go:117] "RemoveContainer" containerID="a73bc749ede0bd9b417e2342ed451cdc0caabd7ad14f373e00030c0c8a8c9a24"
Jul 09 13:59:08.495711 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:59:08.495674    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd" event={"ID":"8039ae6d-e6a1-49d8-a8f8-771681558f0d","Type":"ContainerStarted","Data":"91ebf3545eafab7000ad9574332b63eadf65a68236238c7da144a1ea047617e3"}
Jul 09 13:59:08.496176 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:59:08.496006    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"
Jul 09 13:59:08.496629 ip-10-0-128-186 kubenswrapper[2579]: I0709 13:59:08.496610    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-589b67687c-9lcgd"
Jul 09 14:00:05.672298 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.672261    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pwth7"]
Jul 09 14:00:05.674689 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.674674    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:05.678382 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.678352    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 14:00:05.678507 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.678404    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 14:00:05.679133 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.679114    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kq44v\""
Jul 09 14:00:05.679544 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.679528    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 14:00:05.680214 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.680198    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 14:00:05.693083 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.693059    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pwth7"]
Jul 09 14:00:05.784531 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.784504    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aaf6e0b7-e1c1-41d0-805a-622653b89228-crio-socket\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:05.784698 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.784547    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:05.784698 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.784571    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:05.784698 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.784656    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aaf6e0b7-e1c1-41d0-805a-622653b89228-data-volume\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:05.784862 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.784708    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdskc\" (UniqueName: \"kubernetes.io/projected/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-api-access-cdskc\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:05.885333 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.885294    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:05.885469 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.885358    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aaf6e0b7-e1c1-41d0-805a-622653b89228-data-volume\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:05.885469 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.885401    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdskc\" (UniqueName: \"kubernetes.io/projected/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-api-access-cdskc\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:05.885469 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.885442    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aaf6e0b7-e1c1-41d0-805a-622653b89228-crio-socket\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:05.885469 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:05.885453    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 14:00:05.885614 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.885481    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:05.885614 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:05.885520    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls podName:aaf6e0b7-e1c1-41d0-805a-622653b89228 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:06.385500334 +0000 UTC m=+238.167875267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pwth7" (UID: "aaf6e0b7-e1c1-41d0-805a-622653b89228") : secret "insights-runtime-extractor-tls" not found
Jul 09 14:00:05.885614 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:05.885564    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 14:00:05.885614 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.885583    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aaf6e0b7-e1c1-41d0-805a-622653b89228-crio-socket\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:05.885778 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:05.885624    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm podName:aaf6e0b7-e1c1-41d0-805a-622653b89228 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:06.385611721 +0000 UTC m=+238.167986648 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pwth7" (UID: "aaf6e0b7-e1c1-41d0-805a-622653b89228") : configmap "kube-rbac-proxy" not found
Jul 09 14:00:05.885778 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.885695    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aaf6e0b7-e1c1-41d0-805a-622653b89228-data-volume\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:05.893935 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:05.893911    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdskc\" (UniqueName: \"kubernetes.io/projected/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-api-access-cdskc\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:06.389489 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:06.389451    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:06.389489 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:06.389492    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:06.389692 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:06.389573    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 14:00:06.389692 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:06.389613    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 14:00:06.389692 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:06.389650    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm podName:aaf6e0b7-e1c1-41d0-805a-622653b89228 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:07.389633233 +0000 UTC m=+239.172008151 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pwth7" (UID: "aaf6e0b7-e1c1-41d0-805a-622653b89228") : configmap "kube-rbac-proxy" not found
Jul 09 14:00:06.389692 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:06.389668    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls podName:aaf6e0b7-e1c1-41d0-805a-622653b89228 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:07.389656385 +0000 UTC m=+239.172031297 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pwth7" (UID: "aaf6e0b7-e1c1-41d0-805a-622653b89228") : secret "insights-runtime-extractor-tls" not found
Jul 09 14:00:07.397721 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:07.397674    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:07.397721 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:07.397725    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:07.398160 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:07.397828    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 14:00:07.398160 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:07.397852    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 14:00:07.398160 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:07.397893    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm podName:aaf6e0b7-e1c1-41d0-805a-622653b89228 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:09.397877213 +0000 UTC m=+241.180252131 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pwth7" (UID: "aaf6e0b7-e1c1-41d0-805a-622653b89228") : configmap "kube-rbac-proxy" not found
Jul 09 14:00:07.398160 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:07.397906    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls podName:aaf6e0b7-e1c1-41d0-805a-622653b89228 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:09.397900284 +0000 UTC m=+241.180275198 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pwth7" (UID: "aaf6e0b7-e1c1-41d0-805a-622653b89228") : secret "insights-runtime-extractor-tls" not found
Jul 09 14:00:08.447852 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:08.447827    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wr822_950143f5-da10-4fa7-897f-cd547c9932d0/dns-node-resolver/0.log"
Jul 09 14:00:09.411902 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:09.411858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:09.411902 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:09.411904    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:09.412107 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:09.412002    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 14:00:09.412107 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:09.412039    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 14:00:09.412107 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:09.412071    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm podName:aaf6e0b7-e1c1-41d0-805a-622653b89228 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:13.412053087 +0000 UTC m=+245.194428018 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pwth7" (UID: "aaf6e0b7-e1c1-41d0-805a-622653b89228") : configmap "kube-rbac-proxy" not found
Jul 09 14:00:09.412107 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:09.412087    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls podName:aaf6e0b7-e1c1-41d0-805a-622653b89228 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:13.41207959 +0000 UTC m=+245.194454504 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pwth7" (UID: "aaf6e0b7-e1c1-41d0-805a-622653b89228") : secret "insights-runtime-extractor-tls" not found
Jul 09 14:00:09.447820 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:09.447795    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8rxdv_b39fdbde-6632-4711-a42a-67a86d708c58/node-ca/0.log"
Jul 09 14:00:10.047876 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:10.047847    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-456t7_41620e9f-76f0-410a-8b47-d1a33dca6722/serve-healthcheck-canary/0.log"
Jul 09 14:00:13.444006 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:13.443957    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:13.444006 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:13.444010    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:13.444513 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:13.444069    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 14:00:13.444513 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:13.444132    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm podName:aaf6e0b7-e1c1-41d0-805a-622653b89228 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:21.444116849 +0000 UTC m=+253.226491761 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pwth7" (UID: "aaf6e0b7-e1c1-41d0-805a-622653b89228") : configmap "kube-rbac-proxy" not found
Jul 09 14:00:13.444513 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:13.444131    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 14:00:13.444513 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:13.444180    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls podName:aaf6e0b7-e1c1-41d0-805a-622653b89228 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:21.444168845 +0000 UTC m=+253.226543762 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pwth7" (UID: "aaf6e0b7-e1c1-41d0-805a-622653b89228") : secret "insights-runtime-extractor-tls" not found
Jul 09 14:00:20.491341 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:20.491297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs\") pod \"network-metrics-daemon-jjd92\" (UID: \"fb705aa3-dcba-4a7f-ae12-32ccef61e542\") " pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 14:00:20.493726 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:20.493706    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb705aa3-dcba-4a7f-ae12-32ccef61e542-metrics-certs\") pod \"network-metrics-daemon-jjd92\" (UID: \"fb705aa3-dcba-4a7f-ae12-32ccef61e542\") " pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 14:00:20.522518 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:20.522488    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qkthd\""
Jul 09 14:00:20.529947 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:20.529926    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjd92"
Jul 09 14:00:20.651714 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:20.651683    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jjd92"]
Jul 09 14:00:20.654742 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:00:20.654716    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb705aa3_dcba_4a7f_ae12_32ccef61e542.slice/crio-981a2b189eccf2430842503471a433bf4045b20399330bc5355bc57a77ee5e5d WatchSource:0}: Error finding container 981a2b189eccf2430842503471a433bf4045b20399330bc5355bc57a77ee5e5d: Status 404 returned error can't find the container with id 981a2b189eccf2430842503471a433bf4045b20399330bc5355bc57a77ee5e5d
Jul 09 14:00:20.691588 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:20.691557    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jjd92" event={"ID":"fb705aa3-dcba-4a7f-ae12-32ccef61e542","Type":"ContainerStarted","Data":"981a2b189eccf2430842503471a433bf4045b20399330bc5355bc57a77ee5e5d"}
Jul 09 14:00:21.497912 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:21.497863    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:21.497912 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:21.497915    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:21.498546 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:21.498523    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aaf6e0b7-e1c1-41d0-805a-622653b89228-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:21.500349 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:21.500328    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaf6e0b7-e1c1-41d0-805a-622653b89228-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pwth7\" (UID: \"aaf6e0b7-e1c1-41d0-805a-622653b89228\") " pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:21.586179 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:21.586149    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kq44v\""
Jul 09 14:00:21.594233 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:21.594214    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pwth7"
Jul 09 14:00:21.720737 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:21.720711    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pwth7"]
Jul 09 14:00:21.724600 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:00:21.724567    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaf6e0b7_e1c1_41d0_805a_622653b89228.slice/crio-936ac99b26e6530ea5fe3a3aeeb753ab14daf1f9c01525fbed402ab03d766bf5 WatchSource:0}: Error finding container 936ac99b26e6530ea5fe3a3aeeb753ab14daf1f9c01525fbed402ab03d766bf5: Status 404 returned error can't find the container with id 936ac99b26e6530ea5fe3a3aeeb753ab14daf1f9c01525fbed402ab03d766bf5
Jul 09 14:00:22.698530 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:22.698498    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pwth7" event={"ID":"aaf6e0b7-e1c1-41d0-805a-622653b89228","Type":"ContainerStarted","Data":"936ac99b26e6530ea5fe3a3aeeb753ab14daf1f9c01525fbed402ab03d766bf5"}
Jul 09 14:00:23.703282 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:23.703252    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jjd92" event={"ID":"fb705aa3-dcba-4a7f-ae12-32ccef61e542","Type":"ContainerStarted","Data":"ca41a0f496201cb40ec885aa69d1c4b8db052f319055dbc1b72d47277dc2b189"}
Jul 09 14:00:23.703704 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:23.703337    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jjd92" event={"ID":"fb705aa3-dcba-4a7f-ae12-32ccef61e542","Type":"ContainerStarted","Data":"3fca12129c35d11297201dc5e9fa258635d72b95291cf323b0aad52a02bcde38"}
Jul 09 14:00:23.720255 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:23.720213    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jjd92" podStartSLOduration=252.852076698 podStartE2EDuration="4m15.720195796s" totalImagesPullingTime="2.868119098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:23.719310181 +0000 UTC m=+255.501685111" watchObservedRunningTime="2026-07-09 14:00:23.720195796 +0000 UTC m=+255.502570742"
Jul 09 14:00:28.720790 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:28.720738    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pwth7" event={"ID":"aaf6e0b7-e1c1-41d0-805a-622653b89228","Type":"ContainerStarted","Data":"75002c1e25bbe9e9ac85645477b9f2dfe93c99c397a143ecf634eff459ed58b0"}
Jul 09 14:00:28.721166 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:28.720815    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pwth7" event={"ID":"aaf6e0b7-e1c1-41d0-805a-622653b89228","Type":"ContainerStarted","Data":"0fe9d391616f88174f7169040db074d1ce9cfcac12848a234435919eecd2849c"}
Jul 09 14:00:30.137217 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:30.137178    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-b6ctd"]
Jul 09 14:00:30.140411 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:30.140389    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-b6ctd"
Jul 09 14:00:30.143958 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:30.143884    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-nf2qt\""
Jul 09 14:00:30.143958 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:30.143910    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 09 14:00:30.149963 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:30.149943    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-b6ctd"]
Jul 09 14:00:30.155249 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:30.155226    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f85b5eb2-6cae-4366-a270-c3478df155bc-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-b6ctd\" (UID: \"f85b5eb2-6cae-4366-a270-c3478df155bc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-b6ctd"
Jul 09 14:00:30.255799 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:30.255770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f85b5eb2-6cae-4366-a270-c3478df155bc-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-b6ctd\" (UID: \"f85b5eb2-6cae-4366-a270-c3478df155bc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-b6ctd"
Jul 09 14:00:30.258791 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:30.258739    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f85b5eb2-6cae-4366-a270-c3478df155bc-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-b6ctd\" (UID: \"f85b5eb2-6cae-4366-a270-c3478df155bc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-b6ctd"
Jul 09 14:00:30.451407 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:30.451375    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-b6ctd"
Jul 09 14:00:30.597957 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:30.597930    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-b6ctd"]
Jul 09 14:00:30.600896 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:00:30.600867    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf85b5eb2_6cae_4366_a270_c3478df155bc.slice/crio-046de57d0d9a1f4c4c19f3beb97e4d2ef82516eaf2c51af623827c09604429d8 WatchSource:0}: Error finding container 046de57d0d9a1f4c4c19f3beb97e4d2ef82516eaf2c51af623827c09604429d8: Status 404 returned error can't find the container with id 046de57d0d9a1f4c4c19f3beb97e4d2ef82516eaf2c51af623827c09604429d8
Jul 09 14:00:30.729144 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:30.729066    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-b6ctd" event={"ID":"f85b5eb2-6cae-4366-a270-c3478df155bc","Type":"ContainerStarted","Data":"046de57d0d9a1f4c4c19f3beb97e4d2ef82516eaf2c51af623827c09604429d8"}
Jul 09 14:00:32.737001 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:32.736964    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pwth7" event={"ID":"aaf6e0b7-e1c1-41d0-805a-622653b89228","Type":"ContainerStarted","Data":"315f8b5c05be79c8a7a75963619382c3764906f14976bbdbfe794283c8075ec0"}
Jul 09 14:00:32.758698 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:32.758657    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pwth7" podStartSLOduration=17.624229959 podStartE2EDuration="27.758643125s" totalImagesPullingTime="10.134413166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:00:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:32.757484961 +0000 UTC m=+264.539859896" watchObservedRunningTime="2026-07-09 14:00:32.758643125 +0000 UTC m=+264.541018059"
Jul 09 14:00:33.741999 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:33.741962    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-b6ctd" event={"ID":"f85b5eb2-6cae-4366-a270-c3478df155bc","Type":"ContainerStarted","Data":"bfce020be62b0af043f6c56e6c099aad335752c311c5c43c4bf6909ed21322e3"}
Jul 09 14:00:33.757741 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:33.757694    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-b6ctd" podStartSLOduration=1.638866419 podStartE2EDuration="3.757679449s" totalImagesPullingTime="2.11881303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:33.757036939 +0000 UTC m=+265.539411903" watchObservedRunningTime="2026-07-09 14:00:33.757679449 +0000 UTC m=+265.540054384"
Jul 09 14:00:34.745281 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:34.745246    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-b6ctd"
Jul 09 14:00:34.749849 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:34.749827    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-b6ctd"
Jul 09 14:00:41.585797 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.585770    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wrfzs"]
Jul 09 14:00:41.589064 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.589046    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.591460 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.591444    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mkrgj\""
Jul 09 14:00:41.591539 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.591491    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 14:00:41.591734 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.591719    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 14:00:41.591808 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.591781    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 14:00:41.592598 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.592578    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 14:00:41.592598 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.592595    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 14:00:41.592724 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.592619    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 14:00:41.630587 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.630562    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/972019b0-de63-44ce-86c1-8faaa019ef6d-root\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.630740 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.630593    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/972019b0-de63-44ce-86c1-8faaa019ef6d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.630740 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.630613    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/972019b0-de63-44ce-86c1-8faaa019ef6d-node-exporter-accelerators-collector-config\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.630740 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.630642    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/972019b0-de63-44ce-86c1-8faaa019ef6d-node-exporter-tls\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.630740 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.630703    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/972019b0-de63-44ce-86c1-8faaa019ef6d-node-exporter-textfile\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.630740 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.630736    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c8wg\" (UniqueName: \"kubernetes.io/projected/972019b0-de63-44ce-86c1-8faaa019ef6d-kube-api-access-5c8wg\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.630947 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.630779    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/972019b0-de63-44ce-86c1-8faaa019ef6d-metrics-client-ca\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.630947 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.630800    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/972019b0-de63-44ce-86c1-8faaa019ef6d-node-exporter-wtmp\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.630947 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.630816    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/972019b0-de63-44ce-86c1-8faaa019ef6d-sys\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.731969 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.731936    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/972019b0-de63-44ce-86c1-8faaa019ef6d-node-exporter-tls\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.732122 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.731978    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/972019b0-de63-44ce-86c1-8faaa019ef6d-node-exporter-textfile\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.732122 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.732015    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c8wg\" (UniqueName: \"kubernetes.io/projected/972019b0-de63-44ce-86c1-8faaa019ef6d-kube-api-access-5c8wg\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.732122 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.732046    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/972019b0-de63-44ce-86c1-8faaa019ef6d-metrics-client-ca\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.732122 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.732076    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/972019b0-de63-44ce-86c1-8faaa019ef6d-node-exporter-wtmp\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.732122 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.732099    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/972019b0-de63-44ce-86c1-8faaa019ef6d-sys\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.732369 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.732129    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/972019b0-de63-44ce-86c1-8faaa019ef6d-root\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.732369 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.732189    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/972019b0-de63-44ce-86c1-8faaa019ef6d-root\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.732369 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.732200    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/972019b0-de63-44ce-86c1-8faaa019ef6d-sys\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.732369 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.732225    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/972019b0-de63-44ce-86c1-8faaa019ef6d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.732369 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.732258    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/972019b0-de63-44ce-86c1-8faaa019ef6d-node-exporter-accelerators-collector-config\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.732369 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.732250    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/972019b0-de63-44ce-86c1-8faaa019ef6d-node-exporter-wtmp\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.732369 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.732333    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/972019b0-de63-44ce-86c1-8faaa019ef6d-node-exporter-textfile\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.732716 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.732698    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/972019b0-de63-44ce-86c1-8faaa019ef6d-metrics-client-ca\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.732815 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.732799    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/972019b0-de63-44ce-86c1-8faaa019ef6d-node-exporter-accelerators-collector-config\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.734417 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.734392    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/972019b0-de63-44ce-86c1-8faaa019ef6d-node-exporter-tls\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.734579 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.734559    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/972019b0-de63-44ce-86c1-8faaa019ef6d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.741958 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.741940    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c8wg\" (UniqueName: \"kubernetes.io/projected/972019b0-de63-44ce-86c1-8faaa019ef6d-kube-api-access-5c8wg\") pod \"node-exporter-wrfzs\" (UID: \"972019b0-de63-44ce-86c1-8faaa019ef6d\") " pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.897651 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:41.897626    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wrfzs"
Jul 09 14:00:41.907884 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:00:41.907862    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod972019b0_de63_44ce_86c1_8faaa019ef6d.slice/crio-7d56700dcad6678abfcf28e3100c64d7054d3da8bf6c1a7ae330d5615add4425 WatchSource:0}: Error finding container 7d56700dcad6678abfcf28e3100c64d7054d3da8bf6c1a7ae330d5615add4425: Status 404 returned error can't find the container with id 7d56700dcad6678abfcf28e3100c64d7054d3da8bf6c1a7ae330d5615add4425
Jul 09 14:00:42.770760 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:42.770715    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrfzs" event={"ID":"972019b0-de63-44ce-86c1-8faaa019ef6d","Type":"ContainerStarted","Data":"c19c5710687a87bdbc5eb3022a763449269d44e07ce41de12150d62a4153d05c"}
Jul 09 14:00:42.771121 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:42.770799    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrfzs" event={"ID":"972019b0-de63-44ce-86c1-8faaa019ef6d","Type":"ContainerStarted","Data":"7d56700dcad6678abfcf28e3100c64d7054d3da8bf6c1a7ae330d5615add4425"}
Jul 09 14:00:43.774805 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:43.774771    2579 generic.go:356] "Generic (PLEG): container finished" podID="972019b0-de63-44ce-86c1-8faaa019ef6d" containerID="c19c5710687a87bdbc5eb3022a763449269d44e07ce41de12150d62a4153d05c" exitCode=0
Jul 09 14:00:43.775188 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:43.774771    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrfzs" event={"ID":"972019b0-de63-44ce-86c1-8faaa019ef6d","Type":"ContainerDied","Data":"c19c5710687a87bdbc5eb3022a763449269d44e07ce41de12150d62a4153d05c"}
Jul 09 14:00:44.780313 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:44.780280    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrfzs" event={"ID":"972019b0-de63-44ce-86c1-8faaa019ef6d","Type":"ContainerStarted","Data":"0431166c162e5f83eebbc7e5a512d5cf612ab72fa8c8d274fc8d0d051d8cdb99"}
Jul 09 14:00:44.780708 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:44.780336    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrfzs" event={"ID":"972019b0-de63-44ce-86c1-8faaa019ef6d","Type":"ContainerStarted","Data":"106e41667600006e399140a2df4dbc7c4dbd446ddd7977e3820569bec0eda975"}
Jul 09 14:00:44.801669 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:44.801623    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wrfzs" podStartSLOduration=2.9636485930000003 podStartE2EDuration="3.801609966s" totalImagesPullingTime="766.497755ms" totalInitContainerRuntime="71.463618ms" isStatefulPod=true podCreationTimestamp="2026-07-09 14:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:44.801512012 +0000 UTC m=+276.583886947" watchObservedRunningTime="2026-07-09 14:00:44.801609966 +0000 UTC m=+276.583984901"
Jul 09 14:00:45.878802 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:45.878771    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"]
Jul 09 14:00:45.880973 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:45.880959    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:45.885077 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:45.885057    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 09 14:00:45.886144 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:45.886128    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 09 14:00:45.886144 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:45.886138    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4bqm8de344m2r\""
Jul 09 14:00:45.886286 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:45.886199    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-vfbvh\""
Jul 09 14:00:45.886286 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:45.886243    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 09 14:00:45.891836 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:45.891817    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 09 14:00:45.894141 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:45.894118    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"]
Jul 09 14:00:45.965470 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:45.965444    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6gr5\" (UniqueName: \"kubernetes.io/projected/3fa17005-8c04-4cda-a6ff-475400d5a6a0-kube-api-access-j6gr5\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:45.965603 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:45.965473    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3fa17005-8c04-4cda-a6ff-475400d5a6a0-metrics-server-audit-profiles\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:45.965603 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:45.965506    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3fa17005-8c04-4cda-a6ff-475400d5a6a0-secret-metrics-server-tls\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:45.965603 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:45.965551    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3fa17005-8c04-4cda-a6ff-475400d5a6a0-client-ca-bundle\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:45.965603 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:45.965592    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3fa17005-8c04-4cda-a6ff-475400d5a6a0-audit-log\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:45.965841 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:45.965661    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fa17005-8c04-4cda-a6ff-475400d5a6a0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:45.965841 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:45.965706    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3fa17005-8c04-4cda-a6ff-475400d5a6a0-secret-metrics-server-client-certs\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:46.066299 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.066274    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3fa17005-8c04-4cda-a6ff-475400d5a6a0-audit-log\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:46.066416 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.066315    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fa17005-8c04-4cda-a6ff-475400d5a6a0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:46.066416 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.066340    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3fa17005-8c04-4cda-a6ff-475400d5a6a0-secret-metrics-server-client-certs\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:46.066416 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.066363    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6gr5\" (UniqueName: \"kubernetes.io/projected/3fa17005-8c04-4cda-a6ff-475400d5a6a0-kube-api-access-j6gr5\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:46.066416 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.066381    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3fa17005-8c04-4cda-a6ff-475400d5a6a0-metrics-server-audit-profiles\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:46.066416 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.066410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3fa17005-8c04-4cda-a6ff-475400d5a6a0-secret-metrics-server-tls\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:46.066660 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.066423    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3fa17005-8c04-4cda-a6ff-475400d5a6a0-client-ca-bundle\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:46.066712 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.066684    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3fa17005-8c04-4cda-a6ff-475400d5a6a0-audit-log\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:46.067083 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.067060    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fa17005-8c04-4cda-a6ff-475400d5a6a0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:46.067295 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.067277    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3fa17005-8c04-4cda-a6ff-475400d5a6a0-metrics-server-audit-profiles\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:46.068874 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.068855    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3fa17005-8c04-4cda-a6ff-475400d5a6a0-secret-metrics-server-client-certs\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:46.068874 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.068861    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3fa17005-8c04-4cda-a6ff-475400d5a6a0-secret-metrics-server-tls\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:46.068998 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.068893    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3fa17005-8c04-4cda-a6ff-475400d5a6a0-client-ca-bundle\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:46.075828 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.075810    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6gr5\" (UniqueName: \"kubernetes.io/projected/3fa17005-8c04-4cda-a6ff-475400d5a6a0-kube-api-access-j6gr5\") pod \"metrics-server-6999bd6c8b-bc7bj\" (UID: \"3fa17005-8c04-4cda-a6ff-475400d5a6a0\") " pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:46.189682 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.189656    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:00:46.344441 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.344408    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"]
Jul 09 14:00:46.348532 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:00:46.348471    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fa17005_8c04_4cda_a6ff_475400d5a6a0.slice/crio-0506a4b7d9647157b6151d59a49cc4403c1de2b31aa005d26f1ca46d91f111b2 WatchSource:0}: Error finding container 0506a4b7d9647157b6151d59a49cc4403c1de2b31aa005d26f1ca46d91f111b2: Status 404 returned error can't find the container with id 0506a4b7d9647157b6151d59a49cc4403c1de2b31aa005d26f1ca46d91f111b2
Jul 09 14:00:46.787963 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:46.787931    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj" event={"ID":"3fa17005-8c04-4cda-a6ff-475400d5a6a0","Type":"ContainerStarted","Data":"0506a4b7d9647157b6151d59a49cc4403c1de2b31aa005d26f1ca46d91f111b2"}
Jul 09 14:00:47.955124 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.955082    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:00:47.978435 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.978408    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:47.982883 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.982848    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:00:47.983804 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.983781    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 09 14:00:47.984442 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.984268    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 09 14:00:47.984442 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.984277    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 09 14:00:47.984442 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.984315    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 09 14:00:47.984442 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.984381    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 09 14:00:47.984442 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.984271    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9fgbw\""
Jul 09 14:00:47.984805 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.984541    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 09 14:00:47.984805 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.984563    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6h71artpaif40\""
Jul 09 14:00:47.984805 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.984543    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 09 14:00:47.984944 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.984927    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 09 14:00:47.985129 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.985051    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 09 14:00:47.985201 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.985134    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 09 14:00:47.985284 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.985267    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 09 14:00:47.985367 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.985308    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 09 14:00:47.989237 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.989204    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 09 14:00:47.991524 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:47.991430    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 09 14:00:48.083440 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.083410    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.083440 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.083443    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.083659 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.083468    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.083659 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.083535    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.083659 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.083557    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.083659 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.083575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.083659 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.083591    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.083659 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.083608    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/41dea257-68a8-4eb2-8188-d974af017c0d-config-out\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.083659 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.083622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.084000 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.083678    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.084000 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.083720    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8m2j\" (UniqueName: \"kubernetes.io/projected/41dea257-68a8-4eb2-8188-d974af017c0d-kube-api-access-g8m2j\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.084000 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.083811    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.084000 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.083840    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.084000 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.083924    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.084000 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.083968    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.084000 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.083996    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-web-config\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.084357 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.084043    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.084357 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.084098    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.084357 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.084131    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-config\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.084357 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.084156    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/41dea257-68a8-4eb2-8188-d974af017c0d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185203 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185168    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185382 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185219    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185382 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185244    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185382 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185274    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185382 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185296    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185382 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185322    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/41dea257-68a8-4eb2-8188-d974af017c0d-config-out\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185382 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185346    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185382 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185373    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185714 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185395    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8m2j\" (UniqueName: \"kubernetes.io/projected/41dea257-68a8-4eb2-8188-d974af017c0d-kube-api-access-g8m2j\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185714 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185420    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185714 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185450    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185714 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185498    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185714 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185526    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185714 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185552    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-web-config\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185714 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185580    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185714 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185613    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185714 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185640    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-config\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185714 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185666    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/41dea257-68a8-4eb2-8188-d974af017c0d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.185714 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185708    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.186483 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.185732    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.186483 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.186274    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.188216 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.186908    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.188216 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.187029    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.188216 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.187212    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.188216 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.187835    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.188216 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.187847    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.189114 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.189084    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.190229 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.190208    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.190866 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.190843    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.192127 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.192078    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-config\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.192413 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.192387    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-web-config\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.192895 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.192872    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.193001 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.192981    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.193324 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.193302    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/41dea257-68a8-4eb2-8188-d974af017c0d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.193446 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.193419    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.193541 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.193460    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.194003 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.193980    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.194128 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.194099    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.194316 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.194289    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/41dea257-68a8-4eb2-8188-d974af017c0d-config-out\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.195007 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.194987    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8m2j\" (UniqueName: \"kubernetes.io/projected/41dea257-68a8-4eb2-8188-d974af017c0d-kube-api-access-g8m2j\") pod \"prometheus-k8s-0\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.292808 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.292706    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:48.530447 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.530404    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:00:48.667004 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:00:48.666965    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41dea257_68a8_4eb2_8188_d974af017c0d.slice/crio-9b9451fa92cd7e4d58843bc1385670b2cc0c9f5361562514930deb4fe4e90eed WatchSource:0}: Error finding container 9b9451fa92cd7e4d58843bc1385670b2cc0c9f5361562514930deb4fe4e90eed: Status 404 returned error can't find the container with id 9b9451fa92cd7e4d58843bc1385670b2cc0c9f5361562514930deb4fe4e90eed
Jul 09 14:00:48.796554 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:48.796528    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41dea257-68a8-4eb2-8188-d974af017c0d","Type":"ContainerStarted","Data":"9b9451fa92cd7e4d58843bc1385670b2cc0c9f5361562514930deb4fe4e90eed"}
Jul 09 14:00:49.801042 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:49.801003    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj" event={"ID":"3fa17005-8c04-4cda-a6ff-475400d5a6a0","Type":"ContainerStarted","Data":"e596ed974db8d79d388c741f9b19957d6a819a1a7745d7d1de70d11d3d3bad57"}
Jul 09 14:00:49.824036 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:49.823995    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj" podStartSLOduration=2.478052532 podStartE2EDuration="4.823980878s" totalImagesPullingTime="2.345928346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:49.822424727 +0000 UTC m=+281.604799689" watchObservedRunningTime="2026-07-09 14:00:49.823980878 +0000 UTC m=+281.606355813"
Jul 09 14:00:50.434676 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:00:50.434636    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jnpml" podUID="7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9"
Jul 09 14:00:50.806189 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:50.806101    2579 generic.go:356] "Generic (PLEG): container finished" podID="41dea257-68a8-4eb2-8188-d974af017c0d" containerID="8812ac87342703f496d4fe6cad8338e68a484678e09b31389be7baf5963b72e6" exitCode=0
Jul 09 14:00:50.806560 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:50.806193    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41dea257-68a8-4eb2-8188-d974af017c0d","Type":"ContainerDied","Data":"8812ac87342703f496d4fe6cad8338e68a484678e09b31389be7baf5963b72e6"}
Jul 09 14:00:50.806560 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:50.806365    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jnpml"
Jul 09 14:00:53.633041 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:53.633002    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 14:00:53.635660 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:53.635632    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9-metrics-tls\") pod \"dns-default-jnpml\" (UID: \"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9\") " pod="openshift-dns/dns-default-jnpml"
Jul 09 14:00:53.809670 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:53.809635    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dkvmm\""
Jul 09 14:00:53.817112 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:53.817086    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jnpml"
Jul 09 14:00:54.228251 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:54.228209    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jnpml"]
Jul 09 14:00:54.233123 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:00:54.233098    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d0c72fa_1d31_4aa6_b9a4_e1f237fbdfd9.slice/crio-35e70eace796afea96b35bc964f0e318731927c47f981c541f723d2b04f1f8c1 WatchSource:0}: Error finding container 35e70eace796afea96b35bc964f0e318731927c47f981c541f723d2b04f1f8c1: Status 404 returned error can't find the container with id 35e70eace796afea96b35bc964f0e318731927c47f981c541f723d2b04f1f8c1
Jul 09 14:00:54.826464 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:54.826423    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41dea257-68a8-4eb2-8188-d974af017c0d","Type":"ContainerStarted","Data":"398864e404ece458fdd7867aa14a605aac4214e8c9eeb89bd684c1efb8e73c77"}
Jul 09 14:00:54.826855 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:54.826498    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41dea257-68a8-4eb2-8188-d974af017c0d","Type":"ContainerStarted","Data":"650715bef33bb2565e6e8872b2edff80619d0793f1980fa5dac697b2d3929d47"}
Jul 09 14:00:54.826855 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:54.826532    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jnpml" event={"ID":"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9","Type":"ContainerStarted","Data":"35e70eace796afea96b35bc964f0e318731927c47f981c541f723d2b04f1f8c1"}
Jul 09 14:00:56.838230 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:56.838130    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41dea257-68a8-4eb2-8188-d974af017c0d","Type":"ContainerStarted","Data":"7e867008dc2a7212d58e0502d54e9b35ac6b962623ce92d8afd2919f04d58360"}
Jul 09 14:00:56.838230 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:56.838201    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41dea257-68a8-4eb2-8188-d974af017c0d","Type":"ContainerStarted","Data":"ef9325df7d900aa891d5bff79575b1461d14a34a1c9a2455212ebfcba7f98224"}
Jul 09 14:00:56.838711 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:56.838241    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41dea257-68a8-4eb2-8188-d974af017c0d","Type":"ContainerStarted","Data":"ba9dd31aedacb9ab47061c6d1db494b50980841c037912cde20ebe958cb0c814"}
Jul 09 14:00:56.838711 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:56.838281    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41dea257-68a8-4eb2-8188-d974af017c0d","Type":"ContainerStarted","Data":"a294c02b2e72a2d3af730688de614b751440a56221d68d6ea0dc012f1e90cadc"}
Jul 09 14:00:56.839965 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:56.839943    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jnpml" event={"ID":"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9","Type":"ContainerStarted","Data":"4302f2eb95f3b013eba57682f4596b192cd2d8bcf6e8082ed5f92b9601dc9c5b"}
Jul 09 14:00:56.840066 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:56.839980    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jnpml" event={"ID":"7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9","Type":"ContainerStarted","Data":"64b6df913e15e4fb2eb4ddc6a3177c3a61deba06f8747749394bf325f9716bcd"}
Jul 09 14:00:56.840115 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:56.840064    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jnpml"
Jul 09 14:00:56.873822 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:56.873777    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.200952681 podStartE2EDuration="9.873740462s" totalImagesPullingTime="6.651403448s" totalInitContainerRuntime="21.384333ms" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:56.870947511 +0000 UTC m=+288.653322448" watchObservedRunningTime="2026-07-09 14:00:56.873740462 +0000 UTC m=+288.656115395"
Jul 09 14:00:56.892663 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:56.892616    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jnpml" podStartSLOduration=251.7981595 podStartE2EDuration="4m13.892606627s" totalImagesPullingTime="2.094447127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:56.891587584 +0000 UTC m=+288.673962520" watchObservedRunningTime="2026-07-09 14:00:56.892606627 +0000 UTC m=+288.674981559"
Jul 09 14:00:58.293249 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:00:58.293209    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:01.338829 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:01.338791    2579 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb" podUID="fcd91fdb-9ec8-4f4b-828a-95c18bfb087c" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 09 14:01:06.190828 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:06.190789    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:01:06.191253 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:06.190846    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:01:06.846191 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:06.846162    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jnpml"
Jul 09 14:01:08.633072 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:08.633041    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 14:01:08.633983 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:08.633961    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 14:01:08.636983 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:08.636966    2579 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 14:01:11.338367 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:11.338330    2579 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb" podUID="fcd91fdb-9ec8-4f4b-828a-95c18bfb087c" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 09 14:01:21.338757 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:21.338714    2579 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb" podUID="fcd91fdb-9ec8-4f4b-828a-95c18bfb087c" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 09 14:01:21.339139 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:21.338817    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb"
Jul 09 14:01:21.339266 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:21.339239    2579 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"3032c3b06f3af611a7fe5d77ad960629d5fa3f569c1607ea8949be4757c9ced8"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 09 14:01:21.339308 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:21.339286    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb" podUID="fcd91fdb-9ec8-4f4b-828a-95c18bfb087c" containerName="service-proxy" containerID="cri-o://3032c3b06f3af611a7fe5d77ad960629d5fa3f569c1607ea8949be4757c9ced8" gracePeriod=30
Jul 09 14:01:21.932394 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:21.932359    2579 generic.go:356] "Generic (PLEG): container finished" podID="fcd91fdb-9ec8-4f4b-828a-95c18bfb087c" containerID="3032c3b06f3af611a7fe5d77ad960629d5fa3f569c1607ea8949be4757c9ced8" exitCode=2
Jul 09 14:01:21.932629 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:21.932415    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb" event={"ID":"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c","Type":"ContainerDied","Data":"3032c3b06f3af611a7fe5d77ad960629d5fa3f569c1607ea8949be4757c9ced8"}
Jul 09 14:01:21.932629 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:21.932447    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-976c5fb47-cxmkb" event={"ID":"fcd91fdb-9ec8-4f4b-828a-95c18bfb087c","Type":"ContainerStarted","Data":"426918a3fcdf11239a13fdbffbf7fc3574c3c89ea81a858eea3b5b4ad9ec40b5"}
Jul 09 14:01:26.195450 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:26.195421    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:01:26.199387 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:26.199359    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6999bd6c8b-bc7bj"
Jul 09 14:01:48.293603 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:48.293568    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:48.313601 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:48.313578    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:49.042319 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:01:49.042288    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:06.438952 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:06.438908    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:02:06.439458 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:06.439233    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="prometheus" containerID="cri-o://650715bef33bb2565e6e8872b2edff80619d0793f1980fa5dac697b2d3929d47" gracePeriod=600
Jul 09 14:02:06.439458 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:06.439270    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="kube-rbac-proxy" containerID="cri-o://ef9325df7d900aa891d5bff79575b1461d14a34a1c9a2455212ebfcba7f98224" gracePeriod=600
Jul 09 14:02:06.439458 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:06.439301    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7e867008dc2a7212d58e0502d54e9b35ac6b962623ce92d8afd2919f04d58360" gracePeriod=600
Jul 09 14:02:06.439458 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:06.439320    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="config-reloader" containerID="cri-o://398864e404ece458fdd7867aa14a605aac4214e8c9eeb89bd684c1efb8e73c77" gracePeriod=600
Jul 09 14:02:06.439458 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:06.439284    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="thanos-sidecar" containerID="cri-o://a294c02b2e72a2d3af730688de614b751440a56221d68d6ea0dc012f1e90cadc" gracePeriod=600
Jul 09 14:02:06.439458 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:06.439312    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="kube-rbac-proxy-web" containerID="cri-o://ba9dd31aedacb9ab47061c6d1db494b50980841c037912cde20ebe958cb0c814" gracePeriod=600
Jul 09 14:02:07.092884 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.092847    2579 generic.go:356] "Generic (PLEG): container finished" podID="41dea257-68a8-4eb2-8188-d974af017c0d" containerID="7e867008dc2a7212d58e0502d54e9b35ac6b962623ce92d8afd2919f04d58360" exitCode=0
Jul 09 14:02:07.092884 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.092871    2579 generic.go:356] "Generic (PLEG): container finished" podID="41dea257-68a8-4eb2-8188-d974af017c0d" containerID="ef9325df7d900aa891d5bff79575b1461d14a34a1c9a2455212ebfcba7f98224" exitCode=0
Jul 09 14:02:07.092884 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.092877    2579 generic.go:356] "Generic (PLEG): container finished" podID="41dea257-68a8-4eb2-8188-d974af017c0d" containerID="a294c02b2e72a2d3af730688de614b751440a56221d68d6ea0dc012f1e90cadc" exitCode=0
Jul 09 14:02:07.092884 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.092883    2579 generic.go:356] "Generic (PLEG): container finished" podID="41dea257-68a8-4eb2-8188-d974af017c0d" containerID="398864e404ece458fdd7867aa14a605aac4214e8c9eeb89bd684c1efb8e73c77" exitCode=0
Jul 09 14:02:07.092884 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.092888    2579 generic.go:356] "Generic (PLEG): container finished" podID="41dea257-68a8-4eb2-8188-d974af017c0d" containerID="650715bef33bb2565e6e8872b2edff80619d0793f1980fa5dac697b2d3929d47" exitCode=0
Jul 09 14:02:07.093189 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.092918    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41dea257-68a8-4eb2-8188-d974af017c0d","Type":"ContainerDied","Data":"7e867008dc2a7212d58e0502d54e9b35ac6b962623ce92d8afd2919f04d58360"}
Jul 09 14:02:07.093189 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.092979    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41dea257-68a8-4eb2-8188-d974af017c0d","Type":"ContainerDied","Data":"ef9325df7d900aa891d5bff79575b1461d14a34a1c9a2455212ebfcba7f98224"}
Jul 09 14:02:07.093189 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.093006    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41dea257-68a8-4eb2-8188-d974af017c0d","Type":"ContainerDied","Data":"a294c02b2e72a2d3af730688de614b751440a56221d68d6ea0dc012f1e90cadc"}
Jul 09 14:02:07.093189 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.093029    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41dea257-68a8-4eb2-8188-d974af017c0d","Type":"ContainerDied","Data":"398864e404ece458fdd7867aa14a605aac4214e8c9eeb89bd684c1efb8e73c77"}
Jul 09 14:02:07.093189 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.093054    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41dea257-68a8-4eb2-8188-d974af017c0d","Type":"ContainerDied","Data":"650715bef33bb2565e6e8872b2edff80619d0793f1980fa5dac697b2d3929d47"}
Jul 09 14:02:07.683836 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.683811    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:07.763282 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763254    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/41dea257-68a8-4eb2-8188-d974af017c0d-tls-assets\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.763433 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763295    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8m2j\" (UniqueName: \"kubernetes.io/projected/41dea257-68a8-4eb2-8188-d974af017c0d-kube-api-access-g8m2j\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.763433 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763330    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-2\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.763433 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763346    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-trusted-ca-bundle\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.763433 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763380    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.763433 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763409    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-web-config\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.763688 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763438    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-grpc-tls\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.763688 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763469    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-metrics-client-ca\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.763688 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763498    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-0\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.763688 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763532    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.763688 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763558    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-tls\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.763688 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763591    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-1\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.763688 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763625    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-thanos-prometheus-http-client-file\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.763688 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763668    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/41dea257-68a8-4eb2-8188-d974af017c0d-config-out\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.764106 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763694    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-kube-rbac-proxy\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.764106 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763717    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-config\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.764106 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763721    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:02:07.764106 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763763    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-kubelet-serving-ca-bundle\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.764106 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763789    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-metrics-client-certs\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.764106 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763812    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-db\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.764106 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.763836    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-serving-certs-ca-bundle\") pod \"41dea257-68a8-4eb2-8188-d974af017c0d\" (UID: \"41dea257-68a8-4eb2-8188-d974af017c0d\") "
Jul 09 14:02:07.764434 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.764137    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:02:07.764983 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.764546    2579 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-metrics-client-ca\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.764983 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.764571    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.764983 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.764681    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:02:07.765190 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.764939    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:02:07.766956 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.765310    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:02:07.766956 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.765865    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:02:07.766956 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.766453    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:02:07.767168 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.767004    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:02:07.767168 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.767092    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:02:07.768558 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.768530    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:02:07.769271 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.769228    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41dea257-68a8-4eb2-8188-d974af017c0d-kube-api-access-g8m2j" (OuterVolumeSpecName: "kube-api-access-g8m2j") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "kube-api-access-g8m2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:02:07.769366 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.769286    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-config" (OuterVolumeSpecName: "config") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:02:07.769366 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.769322    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:02:07.769489 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.769396    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:02:07.769489 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.769458    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41dea257-68a8-4eb2-8188-d974af017c0d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:02:07.769657 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.769638    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:02:07.769862 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.769845    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:02:07.770678 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.770659    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41dea257-68a8-4eb2-8188-d974af017c0d-config-out" (OuterVolumeSpecName: "config-out") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:02:07.770967 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.770952    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:02:07.779389 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.779366    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-web-config" (OuterVolumeSpecName: "web-config") pod "41dea257-68a8-4eb2-8188-d974af017c0d" (UID: "41dea257-68a8-4eb2-8188-d974af017c0d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:02:07.865573 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865480    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8m2j\" (UniqueName: \"kubernetes.io/projected/41dea257-68a8-4eb2-8188-d974af017c0d-kube-api-access-g8m2j\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865573 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865519    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865573 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865532    2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865573 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865541    2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-web-config\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865573 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865551    2579 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-grpc-tls\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865573 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865559    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865573 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865567    2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865573 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865577    2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865573 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865585    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865971 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865595    2579 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865971 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865604    2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/41dea257-68a8-4eb2-8188-d974af017c0d-config-out\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865971 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865612    2579 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-kube-rbac-proxy\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865971 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865619    2579 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-config\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865971 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865628    2579 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865971 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865636    2579 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/41dea257-68a8-4eb2-8188-d974af017c0d-secret-metrics-client-certs\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865971 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865644    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/41dea257-68a8-4eb2-8188-d974af017c0d-prometheus-k8s-db\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865971 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865652    2579 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41dea257-68a8-4eb2-8188-d974af017c0d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:07.865971 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:07.865660    2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/41dea257-68a8-4eb2-8188-d974af017c0d-tls-assets\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:02:08.099651 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.099619    2579 generic.go:356] "Generic (PLEG): container finished" podID="41dea257-68a8-4eb2-8188-d974af017c0d" containerID="ba9dd31aedacb9ab47061c6d1db494b50980841c037912cde20ebe958cb0c814" exitCode=0
Jul 09 14:02:08.099855 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.099705    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41dea257-68a8-4eb2-8188-d974af017c0d","Type":"ContainerDied","Data":"ba9dd31aedacb9ab47061c6d1db494b50980841c037912cde20ebe958cb0c814"}
Jul 09 14:02:08.099855 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.099791    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41dea257-68a8-4eb2-8188-d974af017c0d","Type":"ContainerDied","Data":"9b9451fa92cd7e4d58843bc1385670b2cc0c9f5361562514930deb4fe4e90eed"}
Jul 09 14:02:08.099855 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.099712    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.099855 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.099821    2579 scope.go:117] "RemoveContainer" containerID="7e867008dc2a7212d58e0502d54e9b35ac6b962623ce92d8afd2919f04d58360"
Jul 09 14:02:08.108644 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.108629    2579 scope.go:117] "RemoveContainer" containerID="ef9325df7d900aa891d5bff79575b1461d14a34a1c9a2455212ebfcba7f98224"
Jul 09 14:02:08.115365 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.115349    2579 scope.go:117] "RemoveContainer" containerID="ba9dd31aedacb9ab47061c6d1db494b50980841c037912cde20ebe958cb0c814"
Jul 09 14:02:08.121589 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.121575    2579 scope.go:117] "RemoveContainer" containerID="a294c02b2e72a2d3af730688de614b751440a56221d68d6ea0dc012f1e90cadc"
Jul 09 14:02:08.124848 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.124829    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:02:08.128216 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.128200    2579 scope.go:117] "RemoveContainer" containerID="398864e404ece458fdd7867aa14a605aac4214e8c9eeb89bd684c1efb8e73c77"
Jul 09 14:02:08.131652 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.131634    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:02:08.134907 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.134889    2579 scope.go:117] "RemoveContainer" containerID="650715bef33bb2565e6e8872b2edff80619d0793f1980fa5dac697b2d3929d47"
Jul 09 14:02:08.141519 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.141502    2579 scope.go:117] "RemoveContainer" containerID="8812ac87342703f496d4fe6cad8338e68a484678e09b31389be7baf5963b72e6"
Jul 09 14:02:08.150191 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.150173    2579 scope.go:117] "RemoveContainer" containerID="7e867008dc2a7212d58e0502d54e9b35ac6b962623ce92d8afd2919f04d58360"
Jul 09 14:02:08.150446 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:02:08.150426    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e867008dc2a7212d58e0502d54e9b35ac6b962623ce92d8afd2919f04d58360\": container with ID starting with 7e867008dc2a7212d58e0502d54e9b35ac6b962623ce92d8afd2919f04d58360 not found: ID does not exist" containerID="7e867008dc2a7212d58e0502d54e9b35ac6b962623ce92d8afd2919f04d58360"
Jul 09 14:02:08.150508 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.150453    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e867008dc2a7212d58e0502d54e9b35ac6b962623ce92d8afd2919f04d58360"} err="failed to get container status \"7e867008dc2a7212d58e0502d54e9b35ac6b962623ce92d8afd2919f04d58360\": rpc error: code = NotFound desc = could not find container \"7e867008dc2a7212d58e0502d54e9b35ac6b962623ce92d8afd2919f04d58360\": container with ID starting with 7e867008dc2a7212d58e0502d54e9b35ac6b962623ce92d8afd2919f04d58360 not found: ID does not exist"
Jul 09 14:02:08.150508 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.150469    2579 scope.go:117] "RemoveContainer" containerID="ef9325df7d900aa891d5bff79575b1461d14a34a1c9a2455212ebfcba7f98224"
Jul 09 14:02:08.150701 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:02:08.150687    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef9325df7d900aa891d5bff79575b1461d14a34a1c9a2455212ebfcba7f98224\": container with ID starting with ef9325df7d900aa891d5bff79575b1461d14a34a1c9a2455212ebfcba7f98224 not found: ID does not exist" containerID="ef9325df7d900aa891d5bff79575b1461d14a34a1c9a2455212ebfcba7f98224"
Jul 09 14:02:08.150811 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.150702    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef9325df7d900aa891d5bff79575b1461d14a34a1c9a2455212ebfcba7f98224"} err="failed to get container status \"ef9325df7d900aa891d5bff79575b1461d14a34a1c9a2455212ebfcba7f98224\": rpc error: code = NotFound desc = could not find container \"ef9325df7d900aa891d5bff79575b1461d14a34a1c9a2455212ebfcba7f98224\": container with ID starting with ef9325df7d900aa891d5bff79575b1461d14a34a1c9a2455212ebfcba7f98224 not found: ID does not exist"
Jul 09 14:02:08.150811 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.150715    2579 scope.go:117] "RemoveContainer" containerID="ba9dd31aedacb9ab47061c6d1db494b50980841c037912cde20ebe958cb0c814"
Jul 09 14:02:08.150970 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:02:08.150953    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba9dd31aedacb9ab47061c6d1db494b50980841c037912cde20ebe958cb0c814\": container with ID starting with ba9dd31aedacb9ab47061c6d1db494b50980841c037912cde20ebe958cb0c814 not found: ID does not exist" containerID="ba9dd31aedacb9ab47061c6d1db494b50980841c037912cde20ebe958cb0c814"
Jul 09 14:02:08.151003 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.150978    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba9dd31aedacb9ab47061c6d1db494b50980841c037912cde20ebe958cb0c814"} err="failed to get container status \"ba9dd31aedacb9ab47061c6d1db494b50980841c037912cde20ebe958cb0c814\": rpc error: code = NotFound desc = could not find container \"ba9dd31aedacb9ab47061c6d1db494b50980841c037912cde20ebe958cb0c814\": container with ID starting with ba9dd31aedacb9ab47061c6d1db494b50980841c037912cde20ebe958cb0c814 not found: ID does not exist"
Jul 09 14:02:08.151003 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.150994    2579 scope.go:117] "RemoveContainer" containerID="a294c02b2e72a2d3af730688de614b751440a56221d68d6ea0dc012f1e90cadc"
Jul 09 14:02:08.151198 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:02:08.151184    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a294c02b2e72a2d3af730688de614b751440a56221d68d6ea0dc012f1e90cadc\": container with ID starting with a294c02b2e72a2d3af730688de614b751440a56221d68d6ea0dc012f1e90cadc not found: ID does not exist" containerID="a294c02b2e72a2d3af730688de614b751440a56221d68d6ea0dc012f1e90cadc"
Jul 09 14:02:08.151247 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.151200    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a294c02b2e72a2d3af730688de614b751440a56221d68d6ea0dc012f1e90cadc"} err="failed to get container status \"a294c02b2e72a2d3af730688de614b751440a56221d68d6ea0dc012f1e90cadc\": rpc error: code = NotFound desc = could not find container \"a294c02b2e72a2d3af730688de614b751440a56221d68d6ea0dc012f1e90cadc\": container with ID starting with a294c02b2e72a2d3af730688de614b751440a56221d68d6ea0dc012f1e90cadc not found: ID does not exist"
Jul 09 14:02:08.151247 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.151211    2579 scope.go:117] "RemoveContainer" containerID="398864e404ece458fdd7867aa14a605aac4214e8c9eeb89bd684c1efb8e73c77"
Jul 09 14:02:08.151421 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:02:08.151403    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"398864e404ece458fdd7867aa14a605aac4214e8c9eeb89bd684c1efb8e73c77\": container with ID starting with 398864e404ece458fdd7867aa14a605aac4214e8c9eeb89bd684c1efb8e73c77 not found: ID does not exist" containerID="398864e404ece458fdd7867aa14a605aac4214e8c9eeb89bd684c1efb8e73c77"
Jul 09 14:02:08.151458 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.151425    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"398864e404ece458fdd7867aa14a605aac4214e8c9eeb89bd684c1efb8e73c77"} err="failed to get container status \"398864e404ece458fdd7867aa14a605aac4214e8c9eeb89bd684c1efb8e73c77\": rpc error: code = NotFound desc = could not find container \"398864e404ece458fdd7867aa14a605aac4214e8c9eeb89bd684c1efb8e73c77\": container with ID starting with 398864e404ece458fdd7867aa14a605aac4214e8c9eeb89bd684c1efb8e73c77 not found: ID does not exist"
Jul 09 14:02:08.151458 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.151439    2579 scope.go:117] "RemoveContainer" containerID="650715bef33bb2565e6e8872b2edff80619d0793f1980fa5dac697b2d3929d47"
Jul 09 14:02:08.151635 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:02:08.151620    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"650715bef33bb2565e6e8872b2edff80619d0793f1980fa5dac697b2d3929d47\": container with ID starting with 650715bef33bb2565e6e8872b2edff80619d0793f1980fa5dac697b2d3929d47 not found: ID does not exist" containerID="650715bef33bb2565e6e8872b2edff80619d0793f1980fa5dac697b2d3929d47"
Jul 09 14:02:08.151671 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.151638    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"650715bef33bb2565e6e8872b2edff80619d0793f1980fa5dac697b2d3929d47"} err="failed to get container status \"650715bef33bb2565e6e8872b2edff80619d0793f1980fa5dac697b2d3929d47\": rpc error: code = NotFound desc = could not find container \"650715bef33bb2565e6e8872b2edff80619d0793f1980fa5dac697b2d3929d47\": container with ID starting with 650715bef33bb2565e6e8872b2edff80619d0793f1980fa5dac697b2d3929d47 not found: ID does not exist"
Jul 09 14:02:08.151671 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.151649    2579 scope.go:117] "RemoveContainer" containerID="8812ac87342703f496d4fe6cad8338e68a484678e09b31389be7baf5963b72e6"
Jul 09 14:02:08.151881 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:02:08.151865    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8812ac87342703f496d4fe6cad8338e68a484678e09b31389be7baf5963b72e6\": container with ID starting with 8812ac87342703f496d4fe6cad8338e68a484678e09b31389be7baf5963b72e6 not found: ID does not exist" containerID="8812ac87342703f496d4fe6cad8338e68a484678e09b31389be7baf5963b72e6"
Jul 09 14:02:08.151933 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.151884    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8812ac87342703f496d4fe6cad8338e68a484678e09b31389be7baf5963b72e6"} err="failed to get container status \"8812ac87342703f496d4fe6cad8338e68a484678e09b31389be7baf5963b72e6\": rpc error: code = NotFound desc = could not find container \"8812ac87342703f496d4fe6cad8338e68a484678e09b31389be7baf5963b72e6\": container with ID starting with 8812ac87342703f496d4fe6cad8338e68a484678e09b31389be7baf5963b72e6 not found: ID does not exist"
Jul 09 14:02:08.169174 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.169144    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:02:08.169395 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.169383    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="kube-rbac-proxy-web"
Jul 09 14:02:08.169432 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.169397    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="kube-rbac-proxy-web"
Jul 09 14:02:08.169432 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.169407    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="thanos-sidecar"
Jul 09 14:02:08.169432 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.169413    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="thanos-sidecar"
Jul 09 14:02:08.169432 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.169421    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="prometheus"
Jul 09 14:02:08.169432 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.169428    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="prometheus"
Jul 09 14:02:08.169572 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.169438    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="config-reloader"
Jul 09 14:02:08.169572 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.169443    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="config-reloader"
Jul 09 14:02:08.169572 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.169450    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="kube-rbac-proxy-thanos"
Jul 09 14:02:08.169572 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.169455    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="kube-rbac-proxy-thanos"
Jul 09 14:02:08.169572 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.169465    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="init-config-reloader"
Jul 09 14:02:08.169572 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.169469    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="init-config-reloader"
Jul 09 14:02:08.169572 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.169476    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="kube-rbac-proxy"
Jul 09 14:02:08.169572 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.169480    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="kube-rbac-proxy"
Jul 09 14:02:08.172622 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.170187    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="prometheus"
Jul 09 14:02:08.172622 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.170212    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="config-reloader"
Jul 09 14:02:08.172622 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.170223    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="thanos-sidecar"
Jul 09 14:02:08.172622 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.170232    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="kube-rbac-proxy"
Jul 09 14:02:08.172622 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.170251    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="kube-rbac-proxy-web"
Jul 09 14:02:08.172622 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.170260    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" containerName="kube-rbac-proxy-thanos"
Jul 09 14:02:08.175108 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.175089    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.177964 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.177936    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 09 14:02:08.178056 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.177977    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 09 14:02:08.178056 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.177950    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 09 14:02:08.178056 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.177979    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9fgbw\""
Jul 09 14:02:08.178354 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.178339    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 09 14:02:08.178449 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.178394    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 09 14:02:08.178605 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.178591    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 09 14:02:08.178664 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.178627    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 09 14:02:08.178664 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.178644    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 09 14:02:08.178758 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.178683    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6h71artpaif40\""
Jul 09 14:02:08.179124 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.179110    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 09 14:02:08.179181 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.179143    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 09 14:02:08.179181 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.179149    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 09 14:02:08.179559 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.179545    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 09 14:02:08.184069 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.184050    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 09 14:02:08.185261 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.185246    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 09 14:02:08.199057 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.199032    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:02:08.268153 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268127    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268253 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268171    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268253 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268193    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268253 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268208    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268253 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268228    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f82c8899-47f3-4f37-9828-4410ee20c388-config-out\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268381 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268253    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f82c8899-47f3-4f37-9828-4410ee20c388-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268381 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268280    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268381 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268316    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268381 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268361    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-config\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268493 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268385    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268493 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268409    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268493 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268425    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268493 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268445    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268493 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268467    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgrql\" (UniqueName: \"kubernetes.io/projected/f82c8899-47f3-4f37-9828-4410ee20c388-kube-api-access-kgrql\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268633 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268492    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268633 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268515    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268633 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268533    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-web-config\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268633 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268558    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268633 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268576    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f82c8899-47f3-4f37-9828-4410ee20c388-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.268633 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.268611    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.369893 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.369825    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.369893 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.369861    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f82c8899-47f3-4f37-9828-4410ee20c388-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.369893 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.369882    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370052 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370002    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370052 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370039    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370125 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370064    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370125 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370233 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370119    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f82c8899-47f3-4f37-9828-4410ee20c388-config-out\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370233 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f82c8899-47f3-4f37-9828-4410ee20c388-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370233 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370172    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370233 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370204    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370408 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370239    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-config\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370408 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370246    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f82c8899-47f3-4f37-9828-4410ee20c388-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370408 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370262    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370408 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370295    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370408 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370323    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370408 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370350    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370408 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370373    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgrql\" (UniqueName: \"kubernetes.io/projected/f82c8899-47f3-4f37-9828-4410ee20c388-kube-api-access-kgrql\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370408 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370406    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370975 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370435    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370975 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370465    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-web-config\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370975 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370696    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.370975 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.370739    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.371614 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.371391    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.372429 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.372401    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.373096 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.373065    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.373274 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.373255    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.373343 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.373321    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-web-config\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.373343 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.373335    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.373869 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.373539    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-config\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.373869 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.373588    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.374094 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.374072    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.374373 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.374335    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f82c8899-47f3-4f37-9828-4410ee20c388-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.374521 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.374372    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.375474 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.375451    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.375567 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.375555    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.376039 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.376017    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f82c8899-47f3-4f37-9828-4410ee20c388-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.376112 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.376051    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f82c8899-47f3-4f37-9828-4410ee20c388-config-out\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.376112 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.376070    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f82c8899-47f3-4f37-9828-4410ee20c388-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.383012 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.382992    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgrql\" (UniqueName: \"kubernetes.io/projected/f82c8899-47f3-4f37-9828-4410ee20c388-kube-api-access-kgrql\") pod \"prometheus-k8s-0\" (UID: \"f82c8899-47f3-4f37-9828-4410ee20c388\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.485270 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.485222    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:08.621816 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.621715    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:02:08.626242 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:02:08.626208    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf82c8899_47f3_4f37_9828_4410ee20c388.slice/crio-0dccca4d26051da6ac3c890048a424df58000ccfd8ee61be16aec48a981ef452 WatchSource:0}: Error finding container 0dccca4d26051da6ac3c890048a424df58000ccfd8ee61be16aec48a981ef452: Status 404 returned error can't find the container with id 0dccca4d26051da6ac3c890048a424df58000ccfd8ee61be16aec48a981ef452
Jul 09 14:02:08.823651 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:08.823611    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41dea257-68a8-4eb2-8188-d974af017c0d" path="/var/lib/kubelet/pods/41dea257-68a8-4eb2-8188-d974af017c0d/volumes"
Jul 09 14:02:09.104332 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:09.104302    2579 generic.go:356] "Generic (PLEG): container finished" podID="f82c8899-47f3-4f37-9828-4410ee20c388" containerID="03733f4918ee06e767427e2110aa26cc4386726e67bff4d9a3b38bd3afa8b18c" exitCode=0
Jul 09 14:02:09.104528 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:09.104388    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f82c8899-47f3-4f37-9828-4410ee20c388","Type":"ContainerDied","Data":"03733f4918ee06e767427e2110aa26cc4386726e67bff4d9a3b38bd3afa8b18c"}
Jul 09 14:02:09.104528 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:09.104458    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f82c8899-47f3-4f37-9828-4410ee20c388","Type":"ContainerStarted","Data":"0dccca4d26051da6ac3c890048a424df58000ccfd8ee61be16aec48a981ef452"}
Jul 09 14:02:10.112021 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:10.111987    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f82c8899-47f3-4f37-9828-4410ee20c388","Type":"ContainerStarted","Data":"59141906cf0d75bc216e6a03f3bf431f14d9c24600f8ec6e43865744ef6f5ba6"}
Jul 09 14:02:10.112368 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:10.112078    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f82c8899-47f3-4f37-9828-4410ee20c388","Type":"ContainerStarted","Data":"71337965bb04a3d48a5ed00d44c4dc29f09b62eed7678491d619e2f2dd65165c"}
Jul 09 14:02:10.112368 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:10.112105    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f82c8899-47f3-4f37-9828-4410ee20c388","Type":"ContainerStarted","Data":"057e989df4828a61a718c60ca34199b5ffe43c220cf8ba941b0240faf94e8c52"}
Jul 09 14:02:10.112368 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:10.112135    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f82c8899-47f3-4f37-9828-4410ee20c388","Type":"ContainerStarted","Data":"1d0e9dc8c72dce497ce81a0ddb21c092f9f315340bf999e3a1714a6f574b6487"}
Jul 09 14:02:10.112368 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:10.112162    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f82c8899-47f3-4f37-9828-4410ee20c388","Type":"ContainerStarted","Data":"183ffaa924f7907a68b68a6a4df01abead6034451a37d059d6841f345665801a"}
Jul 09 14:02:10.112368 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:10.112185    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f82c8899-47f3-4f37-9828-4410ee20c388","Type":"ContainerStarted","Data":"795a750eaeee671ce9fef2a01340689e4efb6e3ef83c9eaaebdf3110afefe48b"}
Jul 09 14:02:10.146274 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:10.146225    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.127583372 podStartE2EDuration="2.146209202s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.62583ms" isStatefulPod=false podCreationTimestamp="2026-07-09 14:02:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:02:10.144008504 +0000 UTC m=+361.926383463" watchObservedRunningTime="2026-07-09 14:02:10.146209202 +0000 UTC m=+361.928584137"
Jul 09 14:02:13.485557 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:02:13.485522    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:03:08.485715 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:03:08.485668    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:03:08.501640 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:03:08.501614    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:03:09.335180 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:03:09.335150    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:04:26.048519 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.048487    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"]
Jul 09 14:04:26.050997 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.050974    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.054146 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.054127    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 09 14:04:26.054260 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.054177    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 09 14:04:26.054385 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.054369    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 09 14:04:26.054704 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.054679    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 09 14:04:26.054872 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.054855    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-djqn2\""
Jul 09 14:04:26.107053 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.107030    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"]
Jul 09 14:04:26.177628 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.177600    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bc7d447-b080-4a56-a248-9419f4bef98f-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d74b946fd-qqm4r\" (UID: \"9bc7d447-b080-4a56-a248-9419f4bef98f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.177791 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.177645    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9bc7d447-b080-4a56-a248-9419f4bef98f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d74b946fd-qqm4r\" (UID: \"9bc7d447-b080-4a56-a248-9419f4bef98f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.177791 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.177697    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9bc7d447-b080-4a56-a248-9419f4bef98f-manifests\") pod \"opendatahub-operator-controller-manager-5d74b946fd-qqm4r\" (UID: \"9bc7d447-b080-4a56-a248-9419f4bef98f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.177791 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.177743    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bc7d447-b080-4a56-a248-9419f4bef98f-tmp\") pod \"opendatahub-operator-controller-manager-5d74b946fd-qqm4r\" (UID: \"9bc7d447-b080-4a56-a248-9419f4bef98f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.177791 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.177783    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4bfg\" (UniqueName: \"kubernetes.io/projected/9bc7d447-b080-4a56-a248-9419f4bef98f-kube-api-access-t4bfg\") pod \"opendatahub-operator-controller-manager-5d74b946fd-qqm4r\" (UID: \"9bc7d447-b080-4a56-a248-9419f4bef98f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.278441 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.278409    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bc7d447-b080-4a56-a248-9419f4bef98f-tmp\") pod \"opendatahub-operator-controller-manager-5d74b946fd-qqm4r\" (UID: \"9bc7d447-b080-4a56-a248-9419f4bef98f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.278441 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.278442    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4bfg\" (UniqueName: \"kubernetes.io/projected/9bc7d447-b080-4a56-a248-9419f4bef98f-kube-api-access-t4bfg\") pod \"opendatahub-operator-controller-manager-5d74b946fd-qqm4r\" (UID: \"9bc7d447-b080-4a56-a248-9419f4bef98f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.278643 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.278564    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bc7d447-b080-4a56-a248-9419f4bef98f-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d74b946fd-qqm4r\" (UID: \"9bc7d447-b080-4a56-a248-9419f4bef98f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.278695 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.278638    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9bc7d447-b080-4a56-a248-9419f4bef98f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d74b946fd-qqm4r\" (UID: \"9bc7d447-b080-4a56-a248-9419f4bef98f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.278695 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.278679    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9bc7d447-b080-4a56-a248-9419f4bef98f-manifests\") pod \"opendatahub-operator-controller-manager-5d74b946fd-qqm4r\" (UID: \"9bc7d447-b080-4a56-a248-9419f4bef98f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.279078 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.279052    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9bc7d447-b080-4a56-a248-9419f4bef98f-manifests\") pod \"opendatahub-operator-controller-manager-5d74b946fd-qqm4r\" (UID: \"9bc7d447-b080-4a56-a248-9419f4bef98f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.280733 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.280715    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bc7d447-b080-4a56-a248-9419f4bef98f-tmp\") pod \"opendatahub-operator-controller-manager-5d74b946fd-qqm4r\" (UID: \"9bc7d447-b080-4a56-a248-9419f4bef98f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.281143 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.281113    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bc7d447-b080-4a56-a248-9419f4bef98f-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d74b946fd-qqm4r\" (UID: \"9bc7d447-b080-4a56-a248-9419f4bef98f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.281230 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.281113    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9bc7d447-b080-4a56-a248-9419f4bef98f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d74b946fd-qqm4r\" (UID: \"9bc7d447-b080-4a56-a248-9419f4bef98f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.290179 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.290159    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4bfg\" (UniqueName: \"kubernetes.io/projected/9bc7d447-b080-4a56-a248-9419f4bef98f-kube-api-access-t4bfg\") pod \"opendatahub-operator-controller-manager-5d74b946fd-qqm4r\" (UID: \"9bc7d447-b080-4a56-a248-9419f4bef98f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.360382 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.360302    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:26.497724 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.497701    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"]
Jul 09 14:04:26.499688 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:04:26.499666    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bc7d447_b080_4a56_a248_9419f4bef98f.slice/crio-d537fd2c561780109022673a3bdb1bc8c00bf73574862f657d63b7fc8d01d48a WatchSource:0}: Error finding container d537fd2c561780109022673a3bdb1bc8c00bf73574862f657d63b7fc8d01d48a: Status 404 returned error can't find the container with id d537fd2c561780109022673a3bdb1bc8c00bf73574862f657d63b7fc8d01d48a
Jul 09 14:04:26.501388 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.501373    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 14:04:26.578267 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:26.578237    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r" event={"ID":"9bc7d447-b080-4a56-a248-9419f4bef98f","Type":"ContainerStarted","Data":"d537fd2c561780109022673a3bdb1bc8c00bf73574862f657d63b7fc8d01d48a"}
Jul 09 14:04:29.597738 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:29.597703    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r" event={"ID":"9bc7d447-b080-4a56-a248-9419f4bef98f","Type":"ContainerStarted","Data":"dd1a30054c27552c80b554f760fbf5bf308404d105301ef2c2de2cd0b411306c"}
Jul 09 14:04:30.602557 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:30.602522    2579 generic.go:356] "Generic (PLEG): container finished" podID="9bc7d447-b080-4a56-a248-9419f4bef98f" containerID="dd1a30054c27552c80b554f760fbf5bf308404d105301ef2c2de2cd0b411306c" exitCode=0
Jul 09 14:04:30.602557 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:30.602563    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r" event={"ID":"9bc7d447-b080-4a56-a248-9419f4bef98f","Type":"ContainerDied","Data":"dd1a30054c27552c80b554f760fbf5bf308404d105301ef2c2de2cd0b411306c"}
Jul 09 14:04:31.610041 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:31.610000    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r" event={"ID":"9bc7d447-b080-4a56-a248-9419f4bef98f","Type":"ContainerStarted","Data":"ba609b6b0ba324d365a24ce1b4d69c1f33dbd7d90828c4a0196c0ac1dddaa54a"}
Jul 09 14:04:31.610483 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:31.610146    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:42.616466 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:42.616433    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r"
Jul 09 14:04:42.641228 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:42.641170    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-qqm4r" podStartSLOduration=12.275397027 podStartE2EDuration="16.641151454s" totalImagesPullingTime="3.301683345s" totalInitContainerRuntime="1.064071082s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:31.640333781 +0000 UTC m=+503.422708717" watchObservedRunningTime="2026-07-09 14:04:42.641151454 +0000 UTC m=+514.423526400"
Jul 09 14:04:44.073764 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.073726    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"]
Jul 09 14:04:44.084066 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.084040    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"
Jul 09 14:04:44.087892 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.087863    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 09 14:04:44.088059 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.087894    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 09 14:04:44.088059 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.087962    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 09 14:04:44.088059 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.087967    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8zgfx\""
Jul 09 14:04:44.088059 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.087867    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 09 14:04:44.088545 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.088525    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 09 14:04:44.089476 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.089458    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"]
Jul 09 14:04:44.119052 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.119022    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfglg\" (UniqueName: \"kubernetes.io/projected/943e2729-26ed-4bd1-a94c-ca9ce3b01cf7-kube-api-access-xfglg\") pod \"lws-controller-manager-c98bd85cf-j8b8f\" (UID: \"943e2729-26ed-4bd1-a94c-ca9ce3b01cf7\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"
Jul 09 14:04:44.119159 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.119055    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/943e2729-26ed-4bd1-a94c-ca9ce3b01cf7-cert\") pod \"lws-controller-manager-c98bd85cf-j8b8f\" (UID: \"943e2729-26ed-4bd1-a94c-ca9ce3b01cf7\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"
Jul 09 14:04:44.119159 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.119082    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/943e2729-26ed-4bd1-a94c-ca9ce3b01cf7-metrics-cert\") pod \"lws-controller-manager-c98bd85cf-j8b8f\" (UID: \"943e2729-26ed-4bd1-a94c-ca9ce3b01cf7\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"
Jul 09 14:04:44.119225 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.119162    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/943e2729-26ed-4bd1-a94c-ca9ce3b01cf7-manager-config\") pod \"lws-controller-manager-c98bd85cf-j8b8f\" (UID: \"943e2729-26ed-4bd1-a94c-ca9ce3b01cf7\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"
Jul 09 14:04:44.219971 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.219937    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/943e2729-26ed-4bd1-a94c-ca9ce3b01cf7-cert\") pod \"lws-controller-manager-c98bd85cf-j8b8f\" (UID: \"943e2729-26ed-4bd1-a94c-ca9ce3b01cf7\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"
Jul 09 14:04:44.219971 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.219976    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/943e2729-26ed-4bd1-a94c-ca9ce3b01cf7-metrics-cert\") pod \"lws-controller-manager-c98bd85cf-j8b8f\" (UID: \"943e2729-26ed-4bd1-a94c-ca9ce3b01cf7\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"
Jul 09 14:04:44.220140 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.220037    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/943e2729-26ed-4bd1-a94c-ca9ce3b01cf7-manager-config\") pod \"lws-controller-manager-c98bd85cf-j8b8f\" (UID: \"943e2729-26ed-4bd1-a94c-ca9ce3b01cf7\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"
Jul 09 14:04:44.220140 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.220108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfglg\" (UniqueName: \"kubernetes.io/projected/943e2729-26ed-4bd1-a94c-ca9ce3b01cf7-kube-api-access-xfglg\") pod \"lws-controller-manager-c98bd85cf-j8b8f\" (UID: \"943e2729-26ed-4bd1-a94c-ca9ce3b01cf7\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"
Jul 09 14:04:44.220788 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.220741    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/943e2729-26ed-4bd1-a94c-ca9ce3b01cf7-manager-config\") pod \"lws-controller-manager-c98bd85cf-j8b8f\" (UID: \"943e2729-26ed-4bd1-a94c-ca9ce3b01cf7\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"
Jul 09 14:04:44.222626 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.222600    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/943e2729-26ed-4bd1-a94c-ca9ce3b01cf7-cert\") pod \"lws-controller-manager-c98bd85cf-j8b8f\" (UID: \"943e2729-26ed-4bd1-a94c-ca9ce3b01cf7\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"
Jul 09 14:04:44.222711 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.222646    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/943e2729-26ed-4bd1-a94c-ca9ce3b01cf7-metrics-cert\") pod \"lws-controller-manager-c98bd85cf-j8b8f\" (UID: \"943e2729-26ed-4bd1-a94c-ca9ce3b01cf7\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"
Jul 09 14:04:44.230703 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.230684    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfglg\" (UniqueName: \"kubernetes.io/projected/943e2729-26ed-4bd1-a94c-ca9ce3b01cf7-kube-api-access-xfglg\") pod \"lws-controller-manager-c98bd85cf-j8b8f\" (UID: \"943e2729-26ed-4bd1-a94c-ca9ce3b01cf7\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"
Jul 09 14:04:44.394130 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.394103    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"
Jul 09 14:04:44.523048 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.523029    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"]
Jul 09 14:04:44.525678 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:04:44.525650    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod943e2729_26ed_4bd1_a94c_ca9ce3b01cf7.slice/crio-a5d6e1952ae42655cd07bb2b5e22049b1221866187dc883b2049fe2593e087d1 WatchSource:0}: Error finding container a5d6e1952ae42655cd07bb2b5e22049b1221866187dc883b2049fe2593e087d1: Status 404 returned error can't find the container with id a5d6e1952ae42655cd07bb2b5e22049b1221866187dc883b2049fe2593e087d1
Jul 09 14:04:44.659168 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:44.659086    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f" event={"ID":"943e2729-26ed-4bd1-a94c-ca9ce3b01cf7","Type":"ContainerStarted","Data":"a5d6e1952ae42655cd07bb2b5e22049b1221866187dc883b2049fe2593e087d1"}
Jul 09 14:04:45.710244 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.710180    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw"]
Jul 09 14:04:45.712868 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.712843    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw"
Jul 09 14:04:45.717979 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.717947    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 09 14:04:45.719313 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.719277    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 09 14:04:45.719557 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.719522    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vmr7x\""
Jul 09 14:04:45.719716 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.719419    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 09 14:04:45.720046 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.719308    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 09 14:04:45.731327 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.731306    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/185a4d54-6cfd-4e0c-a5b6-36c68f8389e5-tls-certs\") pod \"kube-auth-proxy-5f974f5cb9-xb4qw\" (UID: \"185a4d54-6cfd-4e0c-a5b6-36c68f8389e5\") " pod="openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw"
Jul 09 14:04:45.731447 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.731429    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjjvj\" (UniqueName: \"kubernetes.io/projected/185a4d54-6cfd-4e0c-a5b6-36c68f8389e5-kube-api-access-bjjvj\") pod \"kube-auth-proxy-5f974f5cb9-xb4qw\" (UID: \"185a4d54-6cfd-4e0c-a5b6-36c68f8389e5\") " pod="openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw"
Jul 09 14:04:45.731711 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.731610    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/185a4d54-6cfd-4e0c-a5b6-36c68f8389e5-tmp\") pod \"kube-auth-proxy-5f974f5cb9-xb4qw\" (UID: \"185a4d54-6cfd-4e0c-a5b6-36c68f8389e5\") " pod="openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw"
Jul 09 14:04:45.737107 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.737062    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw"]
Jul 09 14:04:45.832976 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.832941    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjjvj\" (UniqueName: \"kubernetes.io/projected/185a4d54-6cfd-4e0c-a5b6-36c68f8389e5-kube-api-access-bjjvj\") pod \"kube-auth-proxy-5f974f5cb9-xb4qw\" (UID: \"185a4d54-6cfd-4e0c-a5b6-36c68f8389e5\") " pod="openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw"
Jul 09 14:04:45.833164 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.833051    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/185a4d54-6cfd-4e0c-a5b6-36c68f8389e5-tmp\") pod \"kube-auth-proxy-5f974f5cb9-xb4qw\" (UID: \"185a4d54-6cfd-4e0c-a5b6-36c68f8389e5\") " pod="openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw"
Jul 09 14:04:45.833164 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.833089    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/185a4d54-6cfd-4e0c-a5b6-36c68f8389e5-tls-certs\") pod \"kube-auth-proxy-5f974f5cb9-xb4qw\" (UID: \"185a4d54-6cfd-4e0c-a5b6-36c68f8389e5\") " pod="openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw"
Jul 09 14:04:45.836147 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.836115    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/185a4d54-6cfd-4e0c-a5b6-36c68f8389e5-tmp\") pod \"kube-auth-proxy-5f974f5cb9-xb4qw\" (UID: \"185a4d54-6cfd-4e0c-a5b6-36c68f8389e5\") " pod="openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw"
Jul 09 14:04:45.836998 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.836972    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/185a4d54-6cfd-4e0c-a5b6-36c68f8389e5-tls-certs\") pod \"kube-auth-proxy-5f974f5cb9-xb4qw\" (UID: \"185a4d54-6cfd-4e0c-a5b6-36c68f8389e5\") " pod="openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw"
Jul 09 14:04:45.855245 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:45.855200    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjjvj\" (UniqueName: \"kubernetes.io/projected/185a4d54-6cfd-4e0c-a5b6-36c68f8389e5-kube-api-access-bjjvj\") pod \"kube-auth-proxy-5f974f5cb9-xb4qw\" (UID: \"185a4d54-6cfd-4e0c-a5b6-36c68f8389e5\") " pod="openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw"
Jul 09 14:04:46.026338 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:46.026020    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw"
Jul 09 14:04:46.251875 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:46.251831    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw"]
Jul 09 14:04:46.261571 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:04:46.261544    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod185a4d54_6cfd_4e0c_a5b6_36c68f8389e5.slice/crio-11eb27ed45032638ef855c343f1c7e90ff583a78bf688c56ee5c5c9de0bfdd3d WatchSource:0}: Error finding container 11eb27ed45032638ef855c343f1c7e90ff583a78bf688c56ee5c5c9de0bfdd3d: Status 404 returned error can't find the container with id 11eb27ed45032638ef855c343f1c7e90ff583a78bf688c56ee5c5c9de0bfdd3d
Jul 09 14:04:46.668138 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:46.668098    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw" event={"ID":"185a4d54-6cfd-4e0c-a5b6-36c68f8389e5","Type":"ContainerStarted","Data":"11eb27ed45032638ef855c343f1c7e90ff583a78bf688c56ee5c5c9de0bfdd3d"}
Jul 09 14:04:47.673268 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:47.673229    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f" event={"ID":"943e2729-26ed-4bd1-a94c-ca9ce3b01cf7","Type":"ContainerStarted","Data":"ec93de9d27a6869f5ef7a67792709f8449085f6e304c14597e80adef081b8b5e"}
Jul 09 14:04:47.673685 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:47.673333    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"
Jul 09 14:04:47.707391 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:47.707336    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f" podStartSLOduration=0.685675318 podStartE2EDuration="3.707316286s" totalImagesPullingTime="3.021640968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:47.704893894 +0000 UTC m=+519.487268830" watchObservedRunningTime="2026-07-09 14:04:47.707316286 +0000 UTC m=+519.489691222"
Jul 09 14:04:50.691296 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:50.691257    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw" event={"ID":"185a4d54-6cfd-4e0c-a5b6-36c68f8389e5","Type":"ContainerStarted","Data":"aa5b25411ab25c1b8aa31ce8ac628028e9c9e597e0993942a30229a571c11de0"}
Jul 09 14:04:58.684106 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:58.684074    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-j8b8f"
Jul 09 14:04:58.742822 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:04:58.742717    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5f974f5cb9-xb4qw" podStartSLOduration=10.346525318 podStartE2EDuration="13.742701591s" totalImagesPullingTime="3.396176273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:50.707953418 +0000 UTC m=+522.490328377" watchObservedRunningTime="2026-07-09 14:04:58.742701591 +0000 UTC m=+530.525076580"
Jul 09 14:06:08.656161 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:06:08.656116    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 14:06:08.656927 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:06:08.656908    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 14:07:16.698136 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:16.698096    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"]
Jul 09 14:07:16.700460 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:16.700436    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"
Jul 09 14:07:16.703498 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:16.703477    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vh7wf\""
Jul 09 14:07:16.703604 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:16.703502    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 14:07:16.704430 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:16.704411    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 09 14:07:16.704538 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:16.704434    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 14:07:16.716218 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:16.716198    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"]
Jul 09 14:07:16.792871 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:16.792836    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"]
Jul 09 14:07:16.884567 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:16.884535    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8775fc4b-f497-4bc2-92e6-0a45ab7299e3-config-file\") pod \"limitador-limitador-64c94fd64f-9cbzw\" (UID: \"8775fc4b-f497-4bc2-92e6-0a45ab7299e3\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"
Jul 09 14:07:16.884735 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:16.884573    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9hwc\" (UniqueName: \"kubernetes.io/projected/8775fc4b-f497-4bc2-92e6-0a45ab7299e3-kube-api-access-j9hwc\") pod \"limitador-limitador-64c94fd64f-9cbzw\" (UID: \"8775fc4b-f497-4bc2-92e6-0a45ab7299e3\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"
Jul 09 14:07:16.985411 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:16.985328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8775fc4b-f497-4bc2-92e6-0a45ab7299e3-config-file\") pod \"limitador-limitador-64c94fd64f-9cbzw\" (UID: \"8775fc4b-f497-4bc2-92e6-0a45ab7299e3\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"
Jul 09 14:07:16.985411 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:16.985365    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9hwc\" (UniqueName: \"kubernetes.io/projected/8775fc4b-f497-4bc2-92e6-0a45ab7299e3-kube-api-access-j9hwc\") pod \"limitador-limitador-64c94fd64f-9cbzw\" (UID: \"8775fc4b-f497-4bc2-92e6-0a45ab7299e3\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"
Jul 09 14:07:16.985954 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:16.985934    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8775fc4b-f497-4bc2-92e6-0a45ab7299e3-config-file\") pod \"limitador-limitador-64c94fd64f-9cbzw\" (UID: \"8775fc4b-f497-4bc2-92e6-0a45ab7299e3\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"
Jul 09 14:07:16.994464 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:16.994431    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9hwc\" (UniqueName: \"kubernetes.io/projected/8775fc4b-f497-4bc2-92e6-0a45ab7299e3-kube-api-access-j9hwc\") pod \"limitador-limitador-64c94fd64f-9cbzw\" (UID: \"8775fc4b-f497-4bc2-92e6-0a45ab7299e3\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"
Jul 09 14:07:17.010254 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:17.010229    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"
Jul 09 14:07:17.143037 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:17.143008    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"]
Jul 09 14:07:17.146482 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:07:17.146453    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8775fc4b_f497_4bc2_92e6_0a45ab7299e3.slice/crio-a958cb97ca6b62d8661f573bd6ed5b693570f59c19048229df07c9f7804db831 WatchSource:0}: Error finding container a958cb97ca6b62d8661f573bd6ed5b693570f59c19048229df07c9f7804db831: Status 404 returned error can't find the container with id a958cb97ca6b62d8661f573bd6ed5b693570f59c19048229df07c9f7804db831
Jul 09 14:07:17.219107 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:17.219072    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw" event={"ID":"8775fc4b-f497-4bc2-92e6-0a45ab7299e3","Type":"ContainerStarted","Data":"a958cb97ca6b62d8661f573bd6ed5b693570f59c19048229df07c9f7804db831"}
Jul 09 14:07:17.539410 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:17.539377    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-8r2w5"]
Jul 09 14:07:17.542372 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:17.542347    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8r2w5"
Jul 09 14:07:17.545258 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:17.545235    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-85j9k\""
Jul 09 14:07:17.553332 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:17.553308    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-8r2w5"]
Jul 09 14:07:17.692011 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:17.691950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6jgp\" (UniqueName: \"kubernetes.io/projected/7abff481-ba0d-491b-abf3-f8f813cb497c-kube-api-access-q6jgp\") pod \"authorino-58cd955675-8r2w5\" (UID: \"7abff481-ba0d-491b-abf3-f8f813cb497c\") " pod="kuadrant-system/authorino-58cd955675-8r2w5"
Jul 09 14:07:17.793288 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:17.793209    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6jgp\" (UniqueName: \"kubernetes.io/projected/7abff481-ba0d-491b-abf3-f8f813cb497c-kube-api-access-q6jgp\") pod \"authorino-58cd955675-8r2w5\" (UID: \"7abff481-ba0d-491b-abf3-f8f813cb497c\") " pod="kuadrant-system/authorino-58cd955675-8r2w5"
Jul 09 14:07:17.805603 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:17.805550    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6jgp\" (UniqueName: \"kubernetes.io/projected/7abff481-ba0d-491b-abf3-f8f813cb497c-kube-api-access-q6jgp\") pod \"authorino-58cd955675-8r2w5\" (UID: \"7abff481-ba0d-491b-abf3-f8f813cb497c\") " pod="kuadrant-system/authorino-58cd955675-8r2w5"
Jul 09 14:07:17.851897 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:17.851858    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8r2w5"
Jul 09 14:07:18.022997 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:18.022945    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-8r2w5"]
Jul 09 14:07:18.025514 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:07:18.025483    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7abff481_ba0d_491b_abf3_f8f813cb497c.slice/crio-29263ac42a260b73bfd23c6d39b94a0af6966e34b7fd48a08c47042f51e9dcb8 WatchSource:0}: Error finding container 29263ac42a260b73bfd23c6d39b94a0af6966e34b7fd48a08c47042f51e9dcb8: Status 404 returned error can't find the container with id 29263ac42a260b73bfd23c6d39b94a0af6966e34b7fd48a08c47042f51e9dcb8
Jul 09 14:07:18.225652 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:18.225607    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8r2w5" event={"ID":"7abff481-ba0d-491b-abf3-f8f813cb497c","Type":"ContainerStarted","Data":"29263ac42a260b73bfd23c6d39b94a0af6966e34b7fd48a08c47042f51e9dcb8"}
Jul 09 14:07:20.235732 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:20.235685    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw" event={"ID":"8775fc4b-f497-4bc2-92e6-0a45ab7299e3","Type":"ContainerStarted","Data":"666ae06ca27800562b57a37cbe1690d7fccd00a0ab0cedb532ebdd4332d9eb5e"}
Jul 09 14:07:20.236238 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:20.235836    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"
Jul 09 14:07:20.256338 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:20.256104    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw" podStartSLOduration=1.649492782 podStartE2EDuration="4.256084395s" totalImagesPullingTime="2.606591613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:20.255452971 +0000 UTC m=+672.037827907" watchObservedRunningTime="2026-07-09 14:07:20.256084395 +0000 UTC m=+672.038459330"
Jul 09 14:07:22.244801 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:22.244734    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8r2w5" event={"ID":"7abff481-ba0d-491b-abf3-f8f813cb497c","Type":"ContainerStarted","Data":"ad565cc53c0042112a38ae1e6b4fc900c060a2395996a622541c287b87ec74f0"}
Jul 09 14:07:22.260848 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:22.260794    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-8r2w5" podStartSLOduration=1.201369487 podStartE2EDuration="5.260774451s" totalImagesPullingTime="4.059404964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:22.260505952 +0000 UTC m=+674.042880888" watchObservedRunningTime="2026-07-09 14:07:22.260774451 +0000 UTC m=+674.043149379"
Jul 09 14:07:22.701992 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:22.701958    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-8r2w5"]
Jul 09 14:07:24.256086 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:24.256020    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-8r2w5" podUID="7abff481-ba0d-491b-abf3-f8f813cb497c" containerName="authorino" containerID="cri-o://ad565cc53c0042112a38ae1e6b4fc900c060a2395996a622541c287b87ec74f0" gracePeriod=30
Jul 09 14:07:24.497080 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:24.497058    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8r2w5"
Jul 09 14:07:24.548308 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:24.548246    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6jgp\" (UniqueName: \"kubernetes.io/projected/7abff481-ba0d-491b-abf3-f8f813cb497c-kube-api-access-q6jgp\") pod \"7abff481-ba0d-491b-abf3-f8f813cb497c\" (UID: \"7abff481-ba0d-491b-abf3-f8f813cb497c\") "
Jul 09 14:07:24.550401 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:24.550373    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7abff481-ba0d-491b-abf3-f8f813cb497c-kube-api-access-q6jgp" (OuterVolumeSpecName: "kube-api-access-q6jgp") pod "7abff481-ba0d-491b-abf3-f8f813cb497c" (UID: "7abff481-ba0d-491b-abf3-f8f813cb497c"). InnerVolumeSpecName "kube-api-access-q6jgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:07:24.648762 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:24.648707    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6jgp\" (UniqueName: \"kubernetes.io/projected/7abff481-ba0d-491b-abf3-f8f813cb497c-kube-api-access-q6jgp\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:07:25.261284 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:25.261248    2579 generic.go:356] "Generic (PLEG): container finished" podID="7abff481-ba0d-491b-abf3-f8f813cb497c" containerID="ad565cc53c0042112a38ae1e6b4fc900c060a2395996a622541c287b87ec74f0" exitCode=0
Jul 09 14:07:25.261284 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:25.261287    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8r2w5" event={"ID":"7abff481-ba0d-491b-abf3-f8f813cb497c","Type":"ContainerDied","Data":"ad565cc53c0042112a38ae1e6b4fc900c060a2395996a622541c287b87ec74f0"}
Jul 09 14:07:25.261773 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:25.261293    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8r2w5"
Jul 09 14:07:25.261773 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:25.261314    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8r2w5" event={"ID":"7abff481-ba0d-491b-abf3-f8f813cb497c","Type":"ContainerDied","Data":"29263ac42a260b73bfd23c6d39b94a0af6966e34b7fd48a08c47042f51e9dcb8"}
Jul 09 14:07:25.261773 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:25.261331    2579 scope.go:117] "RemoveContainer" containerID="ad565cc53c0042112a38ae1e6b4fc900c060a2395996a622541c287b87ec74f0"
Jul 09 14:07:25.269174 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:25.269158    2579 scope.go:117] "RemoveContainer" containerID="ad565cc53c0042112a38ae1e6b4fc900c060a2395996a622541c287b87ec74f0"
Jul 09 14:07:25.269399 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:07:25.269381    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad565cc53c0042112a38ae1e6b4fc900c060a2395996a622541c287b87ec74f0\": container with ID starting with ad565cc53c0042112a38ae1e6b4fc900c060a2395996a622541c287b87ec74f0 not found: ID does not exist" containerID="ad565cc53c0042112a38ae1e6b4fc900c060a2395996a622541c287b87ec74f0"
Jul 09 14:07:25.269445 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:25.269406    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad565cc53c0042112a38ae1e6b4fc900c060a2395996a622541c287b87ec74f0"} err="failed to get container status \"ad565cc53c0042112a38ae1e6b4fc900c060a2395996a622541c287b87ec74f0\": rpc error: code = NotFound desc = could not find container \"ad565cc53c0042112a38ae1e6b4fc900c060a2395996a622541c287b87ec74f0\": container with ID starting with ad565cc53c0042112a38ae1e6b4fc900c060a2395996a622541c287b87ec74f0 not found: ID does not exist"
Jul 09 14:07:25.276905 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:25.276882    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-8r2w5"]
Jul 09 14:07:25.280999 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:25.280981    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-8r2w5"]
Jul 09 14:07:26.824140 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:26.824111    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7abff481-ba0d-491b-abf3-f8f813cb497c" path="/var/lib/kubelet/pods/7abff481-ba0d-491b-abf3-f8f813cb497c/volumes"
Jul 09 14:07:31.241077 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:31.241049    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"
Jul 09 14:07:32.332032 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:32.331993    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"]
Jul 09 14:07:32.332476 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:32.332204    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw" podUID="8775fc4b-f497-4bc2-92e6-0a45ab7299e3" containerName="limitador" containerID="cri-o://666ae06ca27800562b57a37cbe1690d7fccd00a0ab0cedb532ebdd4332d9eb5e" gracePeriod=30
Jul 09 14:07:32.879247 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:32.879227    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"
Jul 09 14:07:32.913705 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:32.913678    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9hwc\" (UniqueName: \"kubernetes.io/projected/8775fc4b-f497-4bc2-92e6-0a45ab7299e3-kube-api-access-j9hwc\") pod \"8775fc4b-f497-4bc2-92e6-0a45ab7299e3\" (UID: \"8775fc4b-f497-4bc2-92e6-0a45ab7299e3\") "
Jul 09 14:07:32.913869 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:32.913734    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8775fc4b-f497-4bc2-92e6-0a45ab7299e3-config-file\") pod \"8775fc4b-f497-4bc2-92e6-0a45ab7299e3\" (UID: \"8775fc4b-f497-4bc2-92e6-0a45ab7299e3\") "
Jul 09 14:07:32.914092 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:32.914072    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8775fc4b-f497-4bc2-92e6-0a45ab7299e3-config-file" (OuterVolumeSpecName: "config-file") pod "8775fc4b-f497-4bc2-92e6-0a45ab7299e3" (UID: "8775fc4b-f497-4bc2-92e6-0a45ab7299e3"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:07:32.915807 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:32.915778    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8775fc4b-f497-4bc2-92e6-0a45ab7299e3-kube-api-access-j9hwc" (OuterVolumeSpecName: "kube-api-access-j9hwc") pod "8775fc4b-f497-4bc2-92e6-0a45ab7299e3" (UID: "8775fc4b-f497-4bc2-92e6-0a45ab7299e3"). InnerVolumeSpecName "kube-api-access-j9hwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:07:33.015094 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:33.015062    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9hwc\" (UniqueName: \"kubernetes.io/projected/8775fc4b-f497-4bc2-92e6-0a45ab7299e3-kube-api-access-j9hwc\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:07:33.015094 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:33.015089    2579 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8775fc4b-f497-4bc2-92e6-0a45ab7299e3-config-file\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:07:33.295816 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:33.295720    2579 generic.go:356] "Generic (PLEG): container finished" podID="8775fc4b-f497-4bc2-92e6-0a45ab7299e3" containerID="666ae06ca27800562b57a37cbe1690d7fccd00a0ab0cedb532ebdd4332d9eb5e" exitCode=0
Jul 09 14:07:33.295816 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:33.295786    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"
Jul 09 14:07:33.295996 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:33.295811    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw" event={"ID":"8775fc4b-f497-4bc2-92e6-0a45ab7299e3","Type":"ContainerDied","Data":"666ae06ca27800562b57a37cbe1690d7fccd00a0ab0cedb532ebdd4332d9eb5e"}
Jul 09 14:07:33.295996 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:33.295870    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-9cbzw" event={"ID":"8775fc4b-f497-4bc2-92e6-0a45ab7299e3","Type":"ContainerDied","Data":"a958cb97ca6b62d8661f573bd6ed5b693570f59c19048229df07c9f7804db831"}
Jul 09 14:07:33.295996 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:33.295899    2579 scope.go:117] "RemoveContainer" containerID="666ae06ca27800562b57a37cbe1690d7fccd00a0ab0cedb532ebdd4332d9eb5e"
Jul 09 14:07:33.303914 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:33.303900    2579 scope.go:117] "RemoveContainer" containerID="666ae06ca27800562b57a37cbe1690d7fccd00a0ab0cedb532ebdd4332d9eb5e"
Jul 09 14:07:33.304119 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:07:33.304101    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"666ae06ca27800562b57a37cbe1690d7fccd00a0ab0cedb532ebdd4332d9eb5e\": container with ID starting with 666ae06ca27800562b57a37cbe1690d7fccd00a0ab0cedb532ebdd4332d9eb5e not found: ID does not exist" containerID="666ae06ca27800562b57a37cbe1690d7fccd00a0ab0cedb532ebdd4332d9eb5e"
Jul 09 14:07:33.304157 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:33.304129    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"666ae06ca27800562b57a37cbe1690d7fccd00a0ab0cedb532ebdd4332d9eb5e"} err="failed to get container status \"666ae06ca27800562b57a37cbe1690d7fccd00a0ab0cedb532ebdd4332d9eb5e\": rpc error: code = NotFound desc = could not find container \"666ae06ca27800562b57a37cbe1690d7fccd00a0ab0cedb532ebdd4332d9eb5e\": container with ID starting with 666ae06ca27800562b57a37cbe1690d7fccd00a0ab0cedb532ebdd4332d9eb5e not found: ID does not exist"
Jul 09 14:07:33.318957 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:33.318925    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"]
Jul 09 14:07:33.320083 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:33.320067    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9cbzw"]
Jul 09 14:07:34.823971 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:34.823933    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8775fc4b-f497-4bc2-92e6-0a45ab7299e3" path="/var/lib/kubelet/pods/8775fc4b-f497-4bc2-92e6-0a45ab7299e3/volumes"
Jul 09 14:07:38.267929 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.267899    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tstml"]
Jul 09 14:07:38.268390 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.268369    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7abff481-ba0d-491b-abf3-f8f813cb497c" containerName="authorino"
Jul 09 14:07:38.268461 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.268395    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7abff481-ba0d-491b-abf3-f8f813cb497c" containerName="authorino"
Jul 09 14:07:38.268461 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.268417    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8775fc4b-f497-4bc2-92e6-0a45ab7299e3" containerName="limitador"
Jul 09 14:07:38.268461 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.268426    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8775fc4b-f497-4bc2-92e6-0a45ab7299e3" containerName="limitador"
Jul 09 14:07:38.268605 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.268494    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8775fc4b-f497-4bc2-92e6-0a45ab7299e3" containerName="limitador"
Jul 09 14:07:38.268605 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.268506    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7abff481-ba0d-491b-abf3-f8f813cb497c" containerName="authorino"
Jul 09 14:07:38.272647 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.272627    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tstml"
Jul 09 14:07:38.280357 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.280336    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 14:07:38.281151 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.281134    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 14:07:38.281151 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.281146    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-k7djw\""
Jul 09 14:07:38.281302 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.281136    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 09 14:07:38.285912 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.285895    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tstml"]
Jul 09 14:07:38.354266 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.354240    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-462v5\" (UniqueName: \"kubernetes.io/projected/f489fce3-7d4f-4917-bb44-88402f209906-kube-api-access-462v5\") pod \"postgres-69fb4594cf-tstml\" (UID: \"f489fce3-7d4f-4917-bb44-88402f209906\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tstml"
Jul 09 14:07:38.354387 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.354326    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f489fce3-7d4f-4917-bb44-88402f209906-data\") pod \"postgres-69fb4594cf-tstml\" (UID: \"f489fce3-7d4f-4917-bb44-88402f209906\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tstml"
Jul 09 14:07:38.455192 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.455167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f489fce3-7d4f-4917-bb44-88402f209906-data\") pod \"postgres-69fb4594cf-tstml\" (UID: \"f489fce3-7d4f-4917-bb44-88402f209906\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tstml"
Jul 09 14:07:38.455296 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.455211    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-462v5\" (UniqueName: \"kubernetes.io/projected/f489fce3-7d4f-4917-bb44-88402f209906-kube-api-access-462v5\") pod \"postgres-69fb4594cf-tstml\" (UID: \"f489fce3-7d4f-4917-bb44-88402f209906\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tstml"
Jul 09 14:07:38.455525 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.455507    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f489fce3-7d4f-4917-bb44-88402f209906-data\") pod \"postgres-69fb4594cf-tstml\" (UID: \"f489fce3-7d4f-4917-bb44-88402f209906\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tstml"
Jul 09 14:07:38.463106 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.463080    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-462v5\" (UniqueName: \"kubernetes.io/projected/f489fce3-7d4f-4917-bb44-88402f209906-kube-api-access-462v5\") pod \"postgres-69fb4594cf-tstml\" (UID: \"f489fce3-7d4f-4917-bb44-88402f209906\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tstml"
Jul 09 14:07:38.581824 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.581725    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tstml"
Jul 09 14:07:38.706162 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:38.706087    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tstml"]
Jul 09 14:07:38.708574 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:07:38.708549    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf489fce3_7d4f_4917_bb44_88402f209906.slice/crio-5f398fbd33ad8a416633571931e970c65cf0e31f4987727b112f7cffa0feb6a2 WatchSource:0}: Error finding container 5f398fbd33ad8a416633571931e970c65cf0e31f4987727b112f7cffa0feb6a2: Status 404 returned error can't find the container with id 5f398fbd33ad8a416633571931e970c65cf0e31f4987727b112f7cffa0feb6a2
Jul 09 14:07:39.321881 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:39.321847    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tstml" event={"ID":"f489fce3-7d4f-4917-bb44-88402f209906","Type":"ContainerStarted","Data":"5f398fbd33ad8a416633571931e970c65cf0e31f4987727b112f7cffa0feb6a2"}
Jul 09 14:07:44.349965 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:44.349925    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tstml" event={"ID":"f489fce3-7d4f-4917-bb44-88402f209906","Type":"ContainerStarted","Data":"71e79d2bf49b3aab9f7ad7cb43168904b53727b924e7dcac4c863f5675c63747"}
Jul 09 14:07:44.350316 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:44.350033    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tstml"
Jul 09 14:07:44.366331 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:44.366285    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tstml" podStartSLOduration=1.359704513 podStartE2EDuration="6.366269771s" totalImagesPullingTime="5.006565258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:44.366048982 +0000 UTC m=+696.148423918" watchObservedRunningTime="2026-07-09 14:07:44.366269771 +0000 UTC m=+696.148644706"
Jul 09 14:07:50.383966 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:50.383929    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tstml"
Jul 09 14:07:50.932323 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:50.932285    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wjzzc"]
Jul 09 14:07:50.937940 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:50.937918    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wjzzc"
Jul 09 14:07:50.940502 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:50.940472    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 14:07:50.940832 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:50.940659    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 14:07:50.940832 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:50.940701    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-85j9k\""
Jul 09 14:07:50.943080 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:50.943062    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wjzzc"]
Jul 09 14:07:51.061540 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.061510    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shpmr\" (UniqueName: \"kubernetes.io/projected/dab9bb7f-0a58-4b36-9c6b-4a5f99774cc3-kube-api-access-shpmr\") pod \"authorino-5d4b67c446-wjzzc\" (UID: \"dab9bb7f-0a58-4b36-9c6b-4a5f99774cc3\") " pod="kuadrant-system/authorino-5d4b67c446-wjzzc"
Jul 09 14:07:51.162609 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.162583    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shpmr\" (UniqueName: \"kubernetes.io/projected/dab9bb7f-0a58-4b36-9c6b-4a5f99774cc3-kube-api-access-shpmr\") pod \"authorino-5d4b67c446-wjzzc\" (UID: \"dab9bb7f-0a58-4b36-9c6b-4a5f99774cc3\") " pod="kuadrant-system/authorino-5d4b67c446-wjzzc"
Jul 09 14:07:51.167699 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.167678    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wjzzc"]
Jul 09 14:07:51.167932 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:07:51.167915    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-shpmr], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-wjzzc" podUID="dab9bb7f-0a58-4b36-9c6b-4a5f99774cc3"
Jul 09 14:07:51.172560 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.172537    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shpmr\" (UniqueName: \"kubernetes.io/projected/dab9bb7f-0a58-4b36-9c6b-4a5f99774cc3-kube-api-access-shpmr\") pod \"authorino-5d4b67c446-wjzzc\" (UID: \"dab9bb7f-0a58-4b36-9c6b-4a5f99774cc3\") " pod="kuadrant-system/authorino-5d4b67c446-wjzzc"
Jul 09 14:07:51.199116 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.199058    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5584cf5b69-vlzzl"]
Jul 09 14:07:51.202647 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.202631    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5584cf5b69-vlzzl"
Jul 09 14:07:51.210909 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.210885    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5584cf5b69-vlzzl"]
Jul 09 14:07:51.263814 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.263787    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mggz\" (UniqueName: \"kubernetes.io/projected/0600df5c-df23-47f0-b7e3-398df2ac9533-kube-api-access-8mggz\") pod \"authorino-5584cf5b69-vlzzl\" (UID: \"0600df5c-df23-47f0-b7e3-398df2ac9533\") " pod="kuadrant-system/authorino-5584cf5b69-vlzzl"
Jul 09 14:07:51.307812 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.307783    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5584cf5b69-vlzzl"]
Jul 09 14:07:51.308046 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:07:51.308028    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-8mggz], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5584cf5b69-vlzzl" podUID="0600df5c-df23-47f0-b7e3-398df2ac9533"
Jul 09 14:07:51.337515 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.337488    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-78774cbb5-gclfc"]
Jul 09 14:07:51.343252 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.342910    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78774cbb5-gclfc"
Jul 09 14:07:51.345763 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.345725    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 09 14:07:51.355935 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.355914    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78774cbb5-gclfc"]
Jul 09 14:07:51.364964 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.364947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mggz\" (UniqueName: \"kubernetes.io/projected/0600df5c-df23-47f0-b7e3-398df2ac9533-kube-api-access-8mggz\") pod \"authorino-5584cf5b69-vlzzl\" (UID: \"0600df5c-df23-47f0-b7e3-398df2ac9533\") " pod="kuadrant-system/authorino-5584cf5b69-vlzzl"
Jul 09 14:07:51.373073 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.373046    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mggz\" (UniqueName: \"kubernetes.io/projected/0600df5c-df23-47f0-b7e3-398df2ac9533-kube-api-access-8mggz\") pod \"authorino-5584cf5b69-vlzzl\" (UID: \"0600df5c-df23-47f0-b7e3-398df2ac9533\") " pod="kuadrant-system/authorino-5584cf5b69-vlzzl"
Jul 09 14:07:51.377948 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.377929    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5584cf5b69-vlzzl"
Jul 09 14:07:51.377948 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.377944    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wjzzc"
Jul 09 14:07:51.385064 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.385046    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5584cf5b69-vlzzl"
Jul 09 14:07:51.387709 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.387696    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wjzzc"
Jul 09 14:07:51.466170 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.466098    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mggz\" (UniqueName: \"kubernetes.io/projected/0600df5c-df23-47f0-b7e3-398df2ac9533-kube-api-access-8mggz\") pod \"0600df5c-df23-47f0-b7e3-398df2ac9533\" (UID: \"0600df5c-df23-47f0-b7e3-398df2ac9533\") "
Jul 09 14:07:51.466170 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.466167    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shpmr\" (UniqueName: \"kubernetes.io/projected/dab9bb7f-0a58-4b36-9c6b-4a5f99774cc3-kube-api-access-shpmr\") pod \"dab9bb7f-0a58-4b36-9c6b-4a5f99774cc3\" (UID: \"dab9bb7f-0a58-4b36-9c6b-4a5f99774cc3\") "
Jul 09 14:07:51.466365 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.466300    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a71464d-c093-4919-a564-a2d75f579fa2-tls-cert\") pod \"authorino-78774cbb5-gclfc\" (UID: \"3a71464d-c093-4919-a564-a2d75f579fa2\") " pod="kuadrant-system/authorino-78774cbb5-gclfc"
Jul 09 14:07:51.466365 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.466338    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqqww\" (UniqueName: \"kubernetes.io/projected/3a71464d-c093-4919-a564-a2d75f579fa2-kube-api-access-wqqww\") pod \"authorino-78774cbb5-gclfc\" (UID: \"3a71464d-c093-4919-a564-a2d75f579fa2\") " pod="kuadrant-system/authorino-78774cbb5-gclfc"
Jul 09 14:07:51.468202 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.468178    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dab9bb7f-0a58-4b36-9c6b-4a5f99774cc3-kube-api-access-shpmr" (OuterVolumeSpecName: "kube-api-access-shpmr") pod "dab9bb7f-0a58-4b36-9c6b-4a5f99774cc3" (UID: "dab9bb7f-0a58-4b36-9c6b-4a5f99774cc3"). InnerVolumeSpecName "kube-api-access-shpmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:07:51.468322 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.468304    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0600df5c-df23-47f0-b7e3-398df2ac9533-kube-api-access-8mggz" (OuterVolumeSpecName: "kube-api-access-8mggz") pod "0600df5c-df23-47f0-b7e3-398df2ac9533" (UID: "0600df5c-df23-47f0-b7e3-398df2ac9533"). InnerVolumeSpecName "kube-api-access-8mggz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:07:51.566966 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.566937    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a71464d-c093-4919-a564-a2d75f579fa2-tls-cert\") pod \"authorino-78774cbb5-gclfc\" (UID: \"3a71464d-c093-4919-a564-a2d75f579fa2\") " pod="kuadrant-system/authorino-78774cbb5-gclfc"
Jul 09 14:07:51.567058 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.566971    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqqww\" (UniqueName: \"kubernetes.io/projected/3a71464d-c093-4919-a564-a2d75f579fa2-kube-api-access-wqqww\") pod \"authorino-78774cbb5-gclfc\" (UID: \"3a71464d-c093-4919-a564-a2d75f579fa2\") " pod="kuadrant-system/authorino-78774cbb5-gclfc"
Jul 09 14:07:51.567108 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.567101    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shpmr\" (UniqueName: \"kubernetes.io/projected/dab9bb7f-0a58-4b36-9c6b-4a5f99774cc3-kube-api-access-shpmr\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:07:51.567251 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.567229    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mggz\" (UniqueName: \"kubernetes.io/projected/0600df5c-df23-47f0-b7e3-398df2ac9533-kube-api-access-8mggz\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:07:51.569363 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.569337    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a71464d-c093-4919-a564-a2d75f579fa2-tls-cert\") pod \"authorino-78774cbb5-gclfc\" (UID: \"3a71464d-c093-4919-a564-a2d75f579fa2\") " pod="kuadrant-system/authorino-78774cbb5-gclfc"
Jul 09 14:07:51.574926 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.574909    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqqww\" (UniqueName: \"kubernetes.io/projected/3a71464d-c093-4919-a564-a2d75f579fa2-kube-api-access-wqqww\") pod \"authorino-78774cbb5-gclfc\" (UID: \"3a71464d-c093-4919-a564-a2d75f579fa2\") " pod="kuadrant-system/authorino-78774cbb5-gclfc"
Jul 09 14:07:51.652164 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.652132    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78774cbb5-gclfc"
Jul 09 14:07:51.774191 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:51.774159    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78774cbb5-gclfc"]
Jul 09 14:07:51.776812 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:07:51.776787    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a71464d_c093_4919_a564_a2d75f579fa2.slice/crio-3a890b2d97ff7188f9e5c973d921a35cca0cef4b452c1a50d524e9b774bcb7f5 WatchSource:0}: Error finding container 3a890b2d97ff7188f9e5c973d921a35cca0cef4b452c1a50d524e9b774bcb7f5: Status 404 returned error can't find the container with id 3a890b2d97ff7188f9e5c973d921a35cca0cef4b452c1a50d524e9b774bcb7f5
Jul 09 14:07:52.383816 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:52.383784    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wjzzc"
Jul 09 14:07:52.383816 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:52.383779    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78774cbb5-gclfc" event={"ID":"3a71464d-c093-4919-a564-a2d75f579fa2","Type":"ContainerStarted","Data":"1500886d73a2bc337b6b29385eb3cc737b5165dde72afaf8c7c34220376eeade"}
Jul 09 14:07:52.384037 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:52.383843    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78774cbb5-gclfc" event={"ID":"3a71464d-c093-4919-a564-a2d75f579fa2","Type":"ContainerStarted","Data":"3a890b2d97ff7188f9e5c973d921a35cca0cef4b452c1a50d524e9b774bcb7f5"}
Jul 09 14:07:52.384093 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:52.384078    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5584cf5b69-vlzzl"
Jul 09 14:07:52.399511 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:52.399455    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-78774cbb5-gclfc" podStartSLOduration=0.965948754 podStartE2EDuration="1.399437892s" totalImagesPullingTime="433.489138ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:52.399029996 +0000 UTC m=+704.181404932" watchObservedRunningTime="2026-07-09 14:07:52.399437892 +0000 UTC m=+704.181812829"
Jul 09 14:07:52.422886 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:52.422863    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5584cf5b69-vlzzl"]
Jul 09 14:07:52.432215 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:52.432185    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5584cf5b69-vlzzl"]
Jul 09 14:07:52.456414 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:52.456385    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wjzzc"]
Jul 09 14:07:52.458819 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:52.458795    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wjzzc"]
Jul 09 14:07:52.824570 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:52.824536    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0600df5c-df23-47f0-b7e3-398df2ac9533" path="/var/lib/kubelet/pods/0600df5c-df23-47f0-b7e3-398df2ac9533/volumes"
Jul 09 14:07:52.824773 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:07:52.824760    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dab9bb7f-0a58-4b36-9c6b-4a5f99774cc3" path="/var/lib/kubelet/pods/dab9bb7f-0a58-4b36-9c6b-4a5f99774cc3/volumes"
Jul 09 14:08:36.741222 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:36.741187    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"]
Jul 09 14:08:36.743590 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:36.743574    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"
Jul 09 14:08:36.746260 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:36.746238    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 09 14:08:36.746422 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:36.746404    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-thdg4\""
Jul 09 14:08:36.754220 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:36.754188    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"]
Jul 09 14:08:36.840591 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:36.840557    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad07dc5c-e700-4390-afa4-e68f342c4e13-maas-api-tls\") pod \"maas-api-7dfbd47d6-9gkf7\" (UID: \"ad07dc5c-e700-4390-afa4-e68f342c4e13\") " pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"
Jul 09 14:08:36.840737 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:36.840607    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad07dc5c-e700-4390-afa4-e68f342c4e13-openshift-service-ca\") pod \"maas-api-7dfbd47d6-9gkf7\" (UID: \"ad07dc5c-e700-4390-afa4-e68f342c4e13\") " pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"
Jul 09 14:08:36.840737 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:36.840704    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltnnm\" (UniqueName: \"kubernetes.io/projected/ad07dc5c-e700-4390-afa4-e68f342c4e13-kube-api-access-ltnnm\") pod \"maas-api-7dfbd47d6-9gkf7\" (UID: \"ad07dc5c-e700-4390-afa4-e68f342c4e13\") " pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"
Jul 09 14:08:36.941506 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:36.941470    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltnnm\" (UniqueName: \"kubernetes.io/projected/ad07dc5c-e700-4390-afa4-e68f342c4e13-kube-api-access-ltnnm\") pod \"maas-api-7dfbd47d6-9gkf7\" (UID: \"ad07dc5c-e700-4390-afa4-e68f342c4e13\") " pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"
Jul 09 14:08:36.941666 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:36.941522    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad07dc5c-e700-4390-afa4-e68f342c4e13-maas-api-tls\") pod \"maas-api-7dfbd47d6-9gkf7\" (UID: \"ad07dc5c-e700-4390-afa4-e68f342c4e13\") " pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"
Jul 09 14:08:36.941666 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:36.941554    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad07dc5c-e700-4390-afa4-e68f342c4e13-openshift-service-ca\") pod \"maas-api-7dfbd47d6-9gkf7\" (UID: \"ad07dc5c-e700-4390-afa4-e68f342c4e13\") " pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"
Jul 09 14:08:36.941736 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:08:36.941684    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 09 14:08:36.941812 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:08:36.941779    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad07dc5c-e700-4390-afa4-e68f342c4e13-maas-api-tls podName:ad07dc5c-e700-4390-afa4-e68f342c4e13 nodeName:}" failed. No retries permitted until 2026-07-09 14:08:37.441739564 +0000 UTC m=+749.224114479 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ad07dc5c-e700-4390-afa4-e68f342c4e13-maas-api-tls") pod "maas-api-7dfbd47d6-9gkf7" (UID: "ad07dc5c-e700-4390-afa4-e68f342c4e13") : secret "maas-api-serving-cert" not found
Jul 09 14:08:36.942211 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:36.942194    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad07dc5c-e700-4390-afa4-e68f342c4e13-openshift-service-ca\") pod \"maas-api-7dfbd47d6-9gkf7\" (UID: \"ad07dc5c-e700-4390-afa4-e68f342c4e13\") " pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"
Jul 09 14:08:36.951166 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:36.951144    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltnnm\" (UniqueName: \"kubernetes.io/projected/ad07dc5c-e700-4390-afa4-e68f342c4e13-kube-api-access-ltnnm\") pod \"maas-api-7dfbd47d6-9gkf7\" (UID: \"ad07dc5c-e700-4390-afa4-e68f342c4e13\") " pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"
Jul 09 14:08:37.446130 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:37.446089    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad07dc5c-e700-4390-afa4-e68f342c4e13-maas-api-tls\") pod \"maas-api-7dfbd47d6-9gkf7\" (UID: \"ad07dc5c-e700-4390-afa4-e68f342c4e13\") " pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"
Jul 09 14:08:37.448567 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:37.448537    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad07dc5c-e700-4390-afa4-e68f342c4e13-maas-api-tls\") pod \"maas-api-7dfbd47d6-9gkf7\" (UID: \"ad07dc5c-e700-4390-afa4-e68f342c4e13\") " pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"
Jul 09 14:08:37.654036 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:37.654006    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"
Jul 09 14:08:37.783654 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:37.783626    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"]
Jul 09 14:08:37.785526 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:08:37.785499    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad07dc5c_e700_4390_afa4_e68f342c4e13.slice/crio-66575d21a41ab643521f334b7a6dbb50ccc4eb8ae5a49522fee879303761a49e WatchSource:0}: Error finding container 66575d21a41ab643521f334b7a6dbb50ccc4eb8ae5a49522fee879303761a49e: Status 404 returned error can't find the container with id 66575d21a41ab643521f334b7a6dbb50ccc4eb8ae5a49522fee879303761a49e
Jul 09 14:08:38.568929 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:38.568895    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7" event={"ID":"ad07dc5c-e700-4390-afa4-e68f342c4e13","Type":"ContainerStarted","Data":"66575d21a41ab643521f334b7a6dbb50ccc4eb8ae5a49522fee879303761a49e"}
Jul 09 14:08:40.578167 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:40.578132    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7" event={"ID":"ad07dc5c-e700-4390-afa4-e68f342c4e13","Type":"ContainerStarted","Data":"45d74dc3c4b105ee5fe00d04bc9883bd062e1b25b6e3fd153ebfd71696074697"}
Jul 09 14:08:40.578554 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:40.578234    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"
Jul 09 14:08:40.596081 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:40.596036    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7" podStartSLOduration=2.421427897 podStartE2EDuration="4.596022961s" totalImagesPullingTime="2.174595064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:08:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:08:40.594200006 +0000 UTC m=+752.376574941" watchObservedRunningTime="2026-07-09 14:08:40.596022961 +0000 UTC m=+752.378397895"
Jul 09 14:08:46.587295 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:08:46.587266    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7dfbd47d6-9gkf7"
Jul 09 14:09:16.147788 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.147738    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"]
Jul 09 14:09:16.150352 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.150333    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.152951 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.152929    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 09 14:09:16.152951 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.152941    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 09 14:09:16.153145 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.152937    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 09 14:09:16.153145 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.152983    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zc5jh\""
Jul 09 14:09:16.166202 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.166184    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"]
Jul 09 14:09:16.174137 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.174116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87d2e702-2781-4a35-9d44-778c95f312d0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.174225 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.174149    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87d2e702-2781-4a35-9d44-778c95f312d0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.174225 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.174180    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87d2e702-2781-4a35-9d44-778c95f312d0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.174305 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.174254    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87d2e702-2781-4a35-9d44-778c95f312d0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.174305 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.174279    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwfvb\" (UniqueName: \"kubernetes.io/projected/87d2e702-2781-4a35-9d44-778c95f312d0-kube-api-access-qwfvb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.174380 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.174304    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87d2e702-2781-4a35-9d44-778c95f312d0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.275495 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.275466    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87d2e702-2781-4a35-9d44-778c95f312d0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.275641 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.275516    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87d2e702-2781-4a35-9d44-778c95f312d0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.275641 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.275534    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwfvb\" (UniqueName: \"kubernetes.io/projected/87d2e702-2781-4a35-9d44-778c95f312d0-kube-api-access-qwfvb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.275641 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.275558    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87d2e702-2781-4a35-9d44-778c95f312d0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.275641 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.275585    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87d2e702-2781-4a35-9d44-778c95f312d0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.275641 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.275610    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87d2e702-2781-4a35-9d44-778c95f312d0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.275919 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.275896    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87d2e702-2781-4a35-9d44-778c95f312d0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.276001 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.275985    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87d2e702-2781-4a35-9d44-778c95f312d0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.276042 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.276015    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87d2e702-2781-4a35-9d44-778c95f312d0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.277874 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.277854    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87d2e702-2781-4a35-9d44-778c95f312d0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.278110 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.278093    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87d2e702-2781-4a35-9d44-778c95f312d0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.284743 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.284718    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwfvb\" (UniqueName: \"kubernetes.io/projected/87d2e702-2781-4a35-9d44-778c95f312d0-kube-api-access-qwfvb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv\" (UID: \"87d2e702-2781-4a35-9d44-778c95f312d0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.463185 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.463098    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:16.602800 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.602742    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"]
Jul 09 14:09:16.607886 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:09:16.607855    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87d2e702_2781_4a35_9d44_778c95f312d0.slice/crio-781153f45bb67e50a99c0cbb0af063a6764e728f2ada27590d7ff41fc06e4d20 WatchSource:0}: Error finding container 781153f45bb67e50a99c0cbb0af063a6764e728f2ada27590d7ff41fc06e4d20: Status 404 returned error can't find the container with id 781153f45bb67e50a99c0cbb0af063a6764e728f2ada27590d7ff41fc06e4d20
Jul 09 14:09:16.721126 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:16.721038    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv" event={"ID":"87d2e702-2781-4a35-9d44-778c95f312d0","Type":"ContainerStarted","Data":"781153f45bb67e50a99c0cbb0af063a6764e728f2ada27590d7ff41fc06e4d20"}
Jul 09 14:09:18.730769 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:18.730712    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv" event={"ID":"87d2e702-2781-4a35-9d44-778c95f312d0","Type":"ContainerStarted","Data":"ae1c9acf596fb49617797c03e0fc66ea05761450a589f61c51722ac76ce781dd"}
Jul 09 14:09:18.750254 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:18.750211    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv" podStartSLOduration=1.64364858 podStartE2EDuration="2.750195335s" totalImagesPullingTime="1.106546755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:09:18.748454089 +0000 UTC m=+790.530829038" watchObservedRunningTime="2026-07-09 14:09:18.750195335 +0000 UTC m=+790.532570269"
Jul 09 14:09:26.464590 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:26.464554    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:26.465191 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:26.464632    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:26.479803 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:26.479777    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:26.780021 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:26.779938    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv"
Jul 09 14:09:55.541382 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.541304    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"]
Jul 09 14:09:55.547742 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.547724    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.552896 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.552875    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 09 14:09:55.558891 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.558870    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"]
Jul 09 14:09:55.717543 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.717508    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05726ee1-9457-4a26-965c-4fcc8d6b8681-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.717701 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.717548    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05726ee1-9457-4a26-965c-4fcc8d6b8681-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.717701 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.717584    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05726ee1-9457-4a26-965c-4fcc8d6b8681-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.717701 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.717641    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05726ee1-9457-4a26-965c-4fcc8d6b8681-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.717701 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.717689    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05726ee1-9457-4a26-965c-4fcc8d6b8681-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.717867 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.717707    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-295gm\" (UniqueName: \"kubernetes.io/projected/05726ee1-9457-4a26-965c-4fcc8d6b8681-kube-api-access-295gm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.818938 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.818868    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05726ee1-9457-4a26-965c-4fcc8d6b8681-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.818938 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.818905    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05726ee1-9457-4a26-965c-4fcc8d6b8681-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.818938 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.818934    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05726ee1-9457-4a26-965c-4fcc8d6b8681-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.819139 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.818959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05726ee1-9457-4a26-965c-4fcc8d6b8681-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.819139 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.819007    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05726ee1-9457-4a26-965c-4fcc8d6b8681-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.819139 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.819030    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-295gm\" (UniqueName: \"kubernetes.io/projected/05726ee1-9457-4a26-965c-4fcc8d6b8681-kube-api-access-295gm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.819385 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.819355    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05726ee1-9457-4a26-965c-4fcc8d6b8681-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.819385 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.819373    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05726ee1-9457-4a26-965c-4fcc8d6b8681-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.819596 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.819435    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05726ee1-9457-4a26-965c-4fcc8d6b8681-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.821265 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.821241    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05726ee1-9457-4a26-965c-4fcc8d6b8681-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.821484 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.821469    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05726ee1-9457-4a26-965c-4fcc8d6b8681-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.827918 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.827899    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-295gm\" (UniqueName: \"kubernetes.io/projected/05726ee1-9457-4a26-965c-4fcc8d6b8681-kube-api-access-295gm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ddm4g\" (UID: \"05726ee1-9457-4a26-965c-4fcc8d6b8681\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.857818 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.857791    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:09:55.990998 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.990691    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"]
Jul 09 14:09:55.995492 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:09:55.995467    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05726ee1_9457_4a26_965c_4fcc8d6b8681.slice/crio-5a32f7bf4aade77e5e9848c80a632a1420c370b9ed882cbca0aec6d265773ad0 WatchSource:0}: Error finding container 5a32f7bf4aade77e5e9848c80a632a1420c370b9ed882cbca0aec6d265773ad0: Status 404 returned error can't find the container with id 5a32f7bf4aade77e5e9848c80a632a1420c370b9ed882cbca0aec6d265773ad0
Jul 09 14:09:55.997045 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:55.997024    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 14:09:56.891073 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:56.891043    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g" event={"ID":"05726ee1-9457-4a26-965c-4fcc8d6b8681","Type":"ContainerStarted","Data":"f29b3e104c14d1499f5b16f40593189fc5b9504d010f5fe2fa3cc93311694113"}
Jul 09 14:09:56.891460 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:56.891091    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g" event={"ID":"05726ee1-9457-4a26-965c-4fcc8d6b8681","Type":"ContainerStarted","Data":"5a32f7bf4aade77e5e9848c80a632a1420c370b9ed882cbca0aec6d265773ad0"}
Jul 09 14:09:56.908799 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:09:56.908727    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g" podStartSLOduration=1.693712019 podStartE2EDuration="1.908712502s" totalImagesPullingTime="215.000483ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:09:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:09:56.90778046 +0000 UTC m=+828.690155393" watchObservedRunningTime="2026-07-09 14:09:56.908712502 +0000 UTC m=+828.691087437"
Jul 09 14:10:05.858807 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:05.858734    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:10:05.859256 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:05.858837    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:10:05.871511 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:05.871484    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:10:05.938598 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:05.938571    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ddm4g"
Jul 09 14:10:07.137281 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.137241    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"]
Jul 09 14:10:07.140449 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.140427    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.143366 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.143347    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 09 14:10:07.149572 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.149551    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"]
Jul 09 14:10:07.217656 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.217630    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/685d7301-539d-4cb1-b2ac-d77ae501b6ba-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.217825 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.217662    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/685d7301-539d-4cb1-b2ac-d77ae501b6ba-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.217825 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.217728    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/685d7301-539d-4cb1-b2ac-d77ae501b6ba-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.217825 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.217760    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/685d7301-539d-4cb1-b2ac-d77ae501b6ba-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.217825 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.217779    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mngnm\" (UniqueName: \"kubernetes.io/projected/685d7301-539d-4cb1-b2ac-d77ae501b6ba-kube-api-access-mngnm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.218011 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.217897    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/685d7301-539d-4cb1-b2ac-d77ae501b6ba-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.318708 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.318674    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/685d7301-539d-4cb1-b2ac-d77ae501b6ba-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.318708 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.318710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/685d7301-539d-4cb1-b2ac-d77ae501b6ba-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.318912 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.318729    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/685d7301-539d-4cb1-b2ac-d77ae501b6ba-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.318912 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.318809    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/685d7301-539d-4cb1-b2ac-d77ae501b6ba-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.318912 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.318827    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/685d7301-539d-4cb1-b2ac-d77ae501b6ba-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.318912 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.318851    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mngnm\" (UniqueName: \"kubernetes.io/projected/685d7301-539d-4cb1-b2ac-d77ae501b6ba-kube-api-access-mngnm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.319123 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.319103    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/685d7301-539d-4cb1-b2ac-d77ae501b6ba-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.319182 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.319142    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/685d7301-539d-4cb1-b2ac-d77ae501b6ba-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.319234 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.319185    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/685d7301-539d-4cb1-b2ac-d77ae501b6ba-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.321254 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.321228    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/685d7301-539d-4cb1-b2ac-d77ae501b6ba-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.321434 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.321416    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/685d7301-539d-4cb1-b2ac-d77ae501b6ba-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.327057 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.327036    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mngnm\" (UniqueName: \"kubernetes.io/projected/685d7301-539d-4cb1-b2ac-d77ae501b6ba-kube-api-access-mngnm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h\" (UID: \"685d7301-539d-4cb1-b2ac-d77ae501b6ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.449892 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.449857    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:07.576093 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.576064    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"]
Jul 09 14:10:07.578398 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:10:07.578371    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod685d7301_539d_4cb1_b2ac_d77ae501b6ba.slice/crio-4c62541da0a1712d3a1d5a11e6693b9ee35a5e6bda23abf7ee94120e7aea19ac WatchSource:0}: Error finding container 4c62541da0a1712d3a1d5a11e6693b9ee35a5e6bda23abf7ee94120e7aea19ac: Status 404 returned error can't find the container with id 4c62541da0a1712d3a1d5a11e6693b9ee35a5e6bda23abf7ee94120e7aea19ac
Jul 09 14:10:07.937072 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.937035    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h" event={"ID":"685d7301-539d-4cb1-b2ac-d77ae501b6ba","Type":"ContainerStarted","Data":"438686158b05a0e24b516fe792fdba771383c45059cda6d64bbd8f93da3a8b9c"}
Jul 09 14:10:07.937212 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.937079    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h" event={"ID":"685d7301-539d-4cb1-b2ac-d77ae501b6ba","Type":"ContainerStarted","Data":"4c62541da0a1712d3a1d5a11e6693b9ee35a5e6bda23abf7ee94120e7aea19ac"}
Jul 09 14:10:07.956116 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:07.956067    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h" podStartSLOduration=0.721133315 podStartE2EDuration="956.050669ms" totalImagesPullingTime="234.917354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:10:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:10:07.954965698 +0000 UTC m=+839.737340635" watchObservedRunningTime="2026-07-09 14:10:07.956050669 +0000 UTC m=+839.738425604"
Jul 09 14:10:17.450214 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:17.450171    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:17.450649 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:17.450475    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:17.462636 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:17.462609    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:10:17.994786 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:10:17.994741    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h"
Jul 09 14:11:06.047018 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:06.046599    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78774cbb5-gclfc"]
Jul 09 14:11:06.047018 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:06.046859    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-78774cbb5-gclfc" podUID="3a71464d-c093-4919-a564-a2d75f579fa2" containerName="authorino" containerID="cri-o://1500886d73a2bc337b6b29385eb3cc737b5165dde72afaf8c7c34220376eeade" gracePeriod=30
Jul 09 14:11:06.185712 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:06.185671    2579 generic.go:356] "Generic (PLEG): container finished" podID="3a71464d-c093-4919-a564-a2d75f579fa2" containerID="1500886d73a2bc337b6b29385eb3cc737b5165dde72afaf8c7c34220376eeade" exitCode=0
Jul 09 14:11:06.185866 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:06.185729    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78774cbb5-gclfc" event={"ID":"3a71464d-c093-4919-a564-a2d75f579fa2","Type":"ContainerDied","Data":"1500886d73a2bc337b6b29385eb3cc737b5165dde72afaf8c7c34220376eeade"}
Jul 09 14:11:06.295708 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:06.295680    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78774cbb5-gclfc"
Jul 09 14:11:06.423175 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:06.423146    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a71464d-c093-4919-a564-a2d75f579fa2-tls-cert\") pod \"3a71464d-c093-4919-a564-a2d75f579fa2\" (UID: \"3a71464d-c093-4919-a564-a2d75f579fa2\") "
Jul 09 14:11:06.423385 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:06.423192    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqqww\" (UniqueName: \"kubernetes.io/projected/3a71464d-c093-4919-a564-a2d75f579fa2-kube-api-access-wqqww\") pod \"3a71464d-c093-4919-a564-a2d75f579fa2\" (UID: \"3a71464d-c093-4919-a564-a2d75f579fa2\") "
Jul 09 14:11:06.425464 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:06.425431    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a71464d-c093-4919-a564-a2d75f579fa2-kube-api-access-wqqww" (OuterVolumeSpecName: "kube-api-access-wqqww") pod "3a71464d-c093-4919-a564-a2d75f579fa2" (UID: "3a71464d-c093-4919-a564-a2d75f579fa2"). InnerVolumeSpecName "kube-api-access-wqqww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:11:06.434826 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:06.434797    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a71464d-c093-4919-a564-a2d75f579fa2-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "3a71464d-c093-4919-a564-a2d75f579fa2" (UID: "3a71464d-c093-4919-a564-a2d75f579fa2"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:11:06.524324 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:06.524295    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3a71464d-c093-4919-a564-a2d75f579fa2-tls-cert\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:11:06.524324 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:06.524319    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqqww\" (UniqueName: \"kubernetes.io/projected/3a71464d-c093-4919-a564-a2d75f579fa2-kube-api-access-wqqww\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:11:07.191798 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:07.191766    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78774cbb5-gclfc" event={"ID":"3a71464d-c093-4919-a564-a2d75f579fa2","Type":"ContainerDied","Data":"3a890b2d97ff7188f9e5c973d921a35cca0cef4b452c1a50d524e9b774bcb7f5"}
Jul 09 14:11:07.191798 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:07.191789    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78774cbb5-gclfc"
Jul 09 14:11:07.192232 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:07.191818    2579 scope.go:117] "RemoveContainer" containerID="1500886d73a2bc337b6b29385eb3cc737b5165dde72afaf8c7c34220376eeade"
Jul 09 14:11:07.213008 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:07.212981    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78774cbb5-gclfc"]
Jul 09 14:11:07.215359 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:07.215341    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-78774cbb5-gclfc"]
Jul 09 14:11:08.682823 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:08.682793    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 14:11:08.683924 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:08.683899    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 14:11:08.825596 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:11:08.825564    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a71464d-c093-4919-a564-a2d75f579fa2" path="/var/lib/kubelet/pods/3a71464d-c093-4919-a564-a2d75f579fa2/volumes"
Jul 09 14:16:08.712872 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:16:08.712843    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 14:16:08.713869 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:16:08.713853    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 14:21:08.742885 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:21:08.742860    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 14:21:08.744707 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:21:08.744686    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 14:24:12.799909 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:12.799877    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"]
Jul 09 14:24:12.800407 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:12.800173    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a71464d-c093-4919-a564-a2d75f579fa2" containerName="authorino"
Jul 09 14:24:12.800407 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:12.800185    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a71464d-c093-4919-a564-a2d75f579fa2" containerName="authorino"
Jul 09 14:24:12.800407 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:12.800238    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a71464d-c093-4919-a564-a2d75f579fa2" containerName="authorino"
Jul 09 14:24:12.803070 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:12.803050    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"
Jul 09 14:24:12.805609 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:12.805587    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b1ce403b-serving-cert\""
Jul 09 14:24:12.816069 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:12.816044    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"]
Jul 09 14:24:12.963802 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:12.963766    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcjts\" (UniqueName: \"kubernetes.io/projected/5461f15b-9aa5-4a68-af07-cda29d33e9bf-kube-api-access-gcjts\") pod \"maas-api-e2e-ait-b1ce403b-75dfd46c-92npb\" (UID: \"5461f15b-9aa5-4a68-af07-cda29d33e9bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"
Jul 09 14:24:12.963996 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:12.963819    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5461f15b-9aa5-4a68-af07-cda29d33e9bf-maas-api-tls\") pod \"maas-api-e2e-ait-b1ce403b-75dfd46c-92npb\" (UID: \"5461f15b-9aa5-4a68-af07-cda29d33e9bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"
Jul 09 14:24:12.963996 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:12.963887    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5461f15b-9aa5-4a68-af07-cda29d33e9bf-openshift-service-ca\") pod \"maas-api-e2e-ait-b1ce403b-75dfd46c-92npb\" (UID: \"5461f15b-9aa5-4a68-af07-cda29d33e9bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"
Jul 09 14:24:13.064507 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:13.064415    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcjts\" (UniqueName: \"kubernetes.io/projected/5461f15b-9aa5-4a68-af07-cda29d33e9bf-kube-api-access-gcjts\") pod \"maas-api-e2e-ait-b1ce403b-75dfd46c-92npb\" (UID: \"5461f15b-9aa5-4a68-af07-cda29d33e9bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"
Jul 09 14:24:13.064507 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:13.064461    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5461f15b-9aa5-4a68-af07-cda29d33e9bf-maas-api-tls\") pod \"maas-api-e2e-ait-b1ce403b-75dfd46c-92npb\" (UID: \"5461f15b-9aa5-4a68-af07-cda29d33e9bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"
Jul 09 14:24:13.064507 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:13.064479    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5461f15b-9aa5-4a68-af07-cda29d33e9bf-openshift-service-ca\") pod \"maas-api-e2e-ait-b1ce403b-75dfd46c-92npb\" (UID: \"5461f15b-9aa5-4a68-af07-cda29d33e9bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"
Jul 09 14:24:13.065191 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:13.065165    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5461f15b-9aa5-4a68-af07-cda29d33e9bf-openshift-service-ca\") pod \"maas-api-e2e-ait-b1ce403b-75dfd46c-92npb\" (UID: \"5461f15b-9aa5-4a68-af07-cda29d33e9bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"
Jul 09 14:24:13.066995 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:13.066976    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5461f15b-9aa5-4a68-af07-cda29d33e9bf-maas-api-tls\") pod \"maas-api-e2e-ait-b1ce403b-75dfd46c-92npb\" (UID: \"5461f15b-9aa5-4a68-af07-cda29d33e9bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"
Jul 09 14:24:13.080766 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:13.080716    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcjts\" (UniqueName: \"kubernetes.io/projected/5461f15b-9aa5-4a68-af07-cda29d33e9bf-kube-api-access-gcjts\") pod \"maas-api-e2e-ait-b1ce403b-75dfd46c-92npb\" (UID: \"5461f15b-9aa5-4a68-af07-cda29d33e9bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"
Jul 09 14:24:13.113533 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:13.113510    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"
Jul 09 14:24:13.257425 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:13.257391    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"]
Jul 09 14:24:13.260228 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:24:13.260198    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5461f15b_9aa5_4a68_af07_cda29d33e9bf.slice/crio-9cf6d61e09a8c92d63e012dcc2004896aa24b39ba81661040f933a62cf3ad88c WatchSource:0}: Error finding container 9cf6d61e09a8c92d63e012dcc2004896aa24b39ba81661040f933a62cf3ad88c: Status 404 returned error can't find the container with id 9cf6d61e09a8c92d63e012dcc2004896aa24b39ba81661040f933a62cf3ad88c
Jul 09 14:24:13.261520 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:13.261504    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 14:24:13.412516 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:13.412484    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb" event={"ID":"5461f15b-9aa5-4a68-af07-cda29d33e9bf","Type":"ContainerStarted","Data":"9cf6d61e09a8c92d63e012dcc2004896aa24b39ba81661040f933a62cf3ad88c"}
Jul 09 14:24:14.419183 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:14.419146    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb" event={"ID":"5461f15b-9aa5-4a68-af07-cda29d33e9bf","Type":"ContainerStarted","Data":"aa6b1fe56692b46241f841f4d0fdbfdb608cc011cfe8186ff95870e999777cd2"}
Jul 09 14:24:14.419536 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:14.419328    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"
Jul 09 14:24:14.467828 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:14.467770    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb" podStartSLOduration=1.377479371 podStartE2EDuration="2.467736828s" totalImagesPullingTime="1.090257457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:24:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:24:14.466118441 +0000 UTC m=+1686.248493375" watchObservedRunningTime="2026-07-09 14:24:14.467736828 +0000 UTC m=+1686.250111764"
Jul 09 14:24:18.386919 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:18.386887    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"]
Jul 09 14:24:18.387465 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:18.387109    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb" podUID="5461f15b-9aa5-4a68-af07-cda29d33e9bf" containerName="maas-api" containerID="cri-o://aa6b1fe56692b46241f841f4d0fdbfdb608cc011cfe8186ff95870e999777cd2" gracePeriod=30
Jul 09 14:24:18.630598 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:18.630575    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"
Jul 09 14:24:18.714451 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:18.714419    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcjts\" (UniqueName: \"kubernetes.io/projected/5461f15b-9aa5-4a68-af07-cda29d33e9bf-kube-api-access-gcjts\") pod \"5461f15b-9aa5-4a68-af07-cda29d33e9bf\" (UID: \"5461f15b-9aa5-4a68-af07-cda29d33e9bf\") "
Jul 09 14:24:18.714647 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:18.714491    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5461f15b-9aa5-4a68-af07-cda29d33e9bf-openshift-service-ca\") pod \"5461f15b-9aa5-4a68-af07-cda29d33e9bf\" (UID: \"5461f15b-9aa5-4a68-af07-cda29d33e9bf\") "
Jul 09 14:24:18.714647 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:18.714523    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5461f15b-9aa5-4a68-af07-cda29d33e9bf-maas-api-tls\") pod \"5461f15b-9aa5-4a68-af07-cda29d33e9bf\" (UID: \"5461f15b-9aa5-4a68-af07-cda29d33e9bf\") "
Jul 09 14:24:18.714871 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:18.714848    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5461f15b-9aa5-4a68-af07-cda29d33e9bf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5461f15b-9aa5-4a68-af07-cda29d33e9bf" (UID: "5461f15b-9aa5-4a68-af07-cda29d33e9bf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:24:18.716839 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:18.716819    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5461f15b-9aa5-4a68-af07-cda29d33e9bf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5461f15b-9aa5-4a68-af07-cda29d33e9bf" (UID: "5461f15b-9aa5-4a68-af07-cda29d33e9bf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:24:18.717047 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:18.717027    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5461f15b-9aa5-4a68-af07-cda29d33e9bf-kube-api-access-gcjts" (OuterVolumeSpecName: "kube-api-access-gcjts") pod "5461f15b-9aa5-4a68-af07-cda29d33e9bf" (UID: "5461f15b-9aa5-4a68-af07-cda29d33e9bf"). InnerVolumeSpecName "kube-api-access-gcjts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:24:18.815332 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:18.815296    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcjts\" (UniqueName: \"kubernetes.io/projected/5461f15b-9aa5-4a68-af07-cda29d33e9bf-kube-api-access-gcjts\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:24:18.815332 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:18.815324    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5461f15b-9aa5-4a68-af07-cda29d33e9bf-openshift-service-ca\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:24:18.815332 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:18.815336    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5461f15b-9aa5-4a68-af07-cda29d33e9bf-maas-api-tls\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:24:19.441637 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:19.441604    2579 generic.go:356] "Generic (PLEG): container finished" podID="5461f15b-9aa5-4a68-af07-cda29d33e9bf" containerID="aa6b1fe56692b46241f841f4d0fdbfdb608cc011cfe8186ff95870e999777cd2" exitCode=0
Jul 09 14:24:19.442097 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:19.441654    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"
Jul 09 14:24:19.442097 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:19.441683    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb" event={"ID":"5461f15b-9aa5-4a68-af07-cda29d33e9bf","Type":"ContainerDied","Data":"aa6b1fe56692b46241f841f4d0fdbfdb608cc011cfe8186ff95870e999777cd2"}
Jul 09 14:24:19.442097 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:19.441779    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb" event={"ID":"5461f15b-9aa5-4a68-af07-cda29d33e9bf","Type":"ContainerDied","Data":"9cf6d61e09a8c92d63e012dcc2004896aa24b39ba81661040f933a62cf3ad88c"}
Jul 09 14:24:19.442097 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:19.441815    2579 scope.go:117] "RemoveContainer" containerID="aa6b1fe56692b46241f841f4d0fdbfdb608cc011cfe8186ff95870e999777cd2"
Jul 09 14:24:19.450083 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:19.450065    2579 scope.go:117] "RemoveContainer" containerID="aa6b1fe56692b46241f841f4d0fdbfdb608cc011cfe8186ff95870e999777cd2"
Jul 09 14:24:19.450330 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:24:19.450309    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa6b1fe56692b46241f841f4d0fdbfdb608cc011cfe8186ff95870e999777cd2\": container with ID starting with aa6b1fe56692b46241f841f4d0fdbfdb608cc011cfe8186ff95870e999777cd2 not found: ID does not exist" containerID="aa6b1fe56692b46241f841f4d0fdbfdb608cc011cfe8186ff95870e999777cd2"
Jul 09 14:24:19.450397 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:19.450342    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa6b1fe56692b46241f841f4d0fdbfdb608cc011cfe8186ff95870e999777cd2"} err="failed to get container status \"aa6b1fe56692b46241f841f4d0fdbfdb608cc011cfe8186ff95870e999777cd2\": rpc error: code = NotFound desc = could not find container \"aa6b1fe56692b46241f841f4d0fdbfdb608cc011cfe8186ff95870e999777cd2\": container with ID starting with aa6b1fe56692b46241f841f4d0fdbfdb608cc011cfe8186ff95870e999777cd2 not found: ID does not exist"
Jul 09 14:24:19.459442 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:19.459418    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"]
Jul 09 14:24:19.462921 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:19.462903    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1ce403b-75dfd46c-92npb"]
Jul 09 14:24:20.824095 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:20.824060    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5461f15b-9aa5-4a68-af07-cda29d33e9bf" path="/var/lib/kubelet/pods/5461f15b-9aa5-4a68-af07-cda29d33e9bf/volumes"
Jul 09 14:24:25.581968 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:25.581936    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"]
Jul 09 14:24:25.582321 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:25.582231    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5461f15b-9aa5-4a68-af07-cda29d33e9bf" containerName="maas-api"
Jul 09 14:24:25.582321 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:25.582241    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5461f15b-9aa5-4a68-af07-cda29d33e9bf" containerName="maas-api"
Jul 09 14:24:25.582321 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:25.582295    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5461f15b-9aa5-4a68-af07-cda29d33e9bf" containerName="maas-api"
Jul 09 14:24:25.586532 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:25.586515    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"
Jul 09 14:24:25.589964 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:25.589937    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f4ec3dfc-serving-cert\""
Jul 09 14:24:25.602027 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:25.602004    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"]
Jul 09 14:24:25.674591 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:25.674552    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c739038f-0d74-40a0-bb6e-9d4e43319971-maas-api-tls\") pod \"maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz\" (UID: \"c739038f-0d74-40a0-bb6e-9d4e43319971\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"
Jul 09 14:24:25.674783 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:25.674624    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tqpm\" (UniqueName: \"kubernetes.io/projected/c739038f-0d74-40a0-bb6e-9d4e43319971-kube-api-access-6tqpm\") pod \"maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz\" (UID: \"c739038f-0d74-40a0-bb6e-9d4e43319971\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"
Jul 09 14:24:25.674783 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:25.674660    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c739038f-0d74-40a0-bb6e-9d4e43319971-openshift-service-ca\") pod \"maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz\" (UID: \"c739038f-0d74-40a0-bb6e-9d4e43319971\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"
Jul 09 14:24:25.775542 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:25.775502    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tqpm\" (UniqueName: \"kubernetes.io/projected/c739038f-0d74-40a0-bb6e-9d4e43319971-kube-api-access-6tqpm\") pod \"maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz\" (UID: \"c739038f-0d74-40a0-bb6e-9d4e43319971\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"
Jul 09 14:24:25.775719 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:25.775555    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c739038f-0d74-40a0-bb6e-9d4e43319971-openshift-service-ca\") pod \"maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz\" (UID: \"c739038f-0d74-40a0-bb6e-9d4e43319971\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"
Jul 09 14:24:25.775719 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:25.775658    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c739038f-0d74-40a0-bb6e-9d4e43319971-maas-api-tls\") pod \"maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz\" (UID: \"c739038f-0d74-40a0-bb6e-9d4e43319971\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"
Jul 09 14:24:25.775849 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:24:25.775782    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-serving-cert: secret "maas-api-e2e-ait-f4ec3dfc-serving-cert" not found
Jul 09 14:24:25.775849 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:24:25.775846    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c739038f-0d74-40a0-bb6e-9d4e43319971-maas-api-tls podName:c739038f-0d74-40a0-bb6e-9d4e43319971 nodeName:}" failed. No retries permitted until 2026-07-09 14:24:26.275826073 +0000 UTC m=+1698.058200999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c739038f-0d74-40a0-bb6e-9d4e43319971-maas-api-tls") pod "maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz" (UID: "c739038f-0d74-40a0-bb6e-9d4e43319971") : secret "maas-api-e2e-ait-f4ec3dfc-serving-cert" not found
Jul 09 14:24:25.776239 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:25.776219    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c739038f-0d74-40a0-bb6e-9d4e43319971-openshift-service-ca\") pod \"maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz\" (UID: \"c739038f-0d74-40a0-bb6e-9d4e43319971\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"
Jul 09 14:24:25.785648 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:25.785629    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tqpm\" (UniqueName: \"kubernetes.io/projected/c739038f-0d74-40a0-bb6e-9d4e43319971-kube-api-access-6tqpm\") pod \"maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz\" (UID: \"c739038f-0d74-40a0-bb6e-9d4e43319971\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"
Jul 09 14:24:26.279466 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:26.279425    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c739038f-0d74-40a0-bb6e-9d4e43319971-maas-api-tls\") pod \"maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz\" (UID: \"c739038f-0d74-40a0-bb6e-9d4e43319971\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"
Jul 09 14:24:26.281979 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:26.281947    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c739038f-0d74-40a0-bb6e-9d4e43319971-maas-api-tls\") pod \"maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz\" (UID: \"c739038f-0d74-40a0-bb6e-9d4e43319971\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"
Jul 09 14:24:26.496304 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:26.496271    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"
Jul 09 14:24:26.629433 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:26.629335    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"]
Jul 09 14:24:26.632003 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:24:26.631977    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc739038f_0d74_40a0_bb6e_9d4e43319971.slice/crio-d13f940439de1b67ab0010ec17f5864226e99ed11f24a3fb3642b9c258c90e4f WatchSource:0}: Error finding container d13f940439de1b67ab0010ec17f5864226e99ed11f24a3fb3642b9c258c90e4f: Status 404 returned error can't find the container with id d13f940439de1b67ab0010ec17f5864226e99ed11f24a3fb3642b9c258c90e4f
Jul 09 14:24:27.476980 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:27.476948    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz" event={"ID":"c739038f-0d74-40a0-bb6e-9d4e43319971","Type":"ContainerStarted","Data":"fef12659969a666ea8e4a7612694577958405609f5a1f2ac35435474a80b0237"}
Jul 09 14:24:27.477150 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:27.476996    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz" event={"ID":"c739038f-0d74-40a0-bb6e-9d4e43319971","Type":"ContainerStarted","Data":"d13f940439de1b67ab0010ec17f5864226e99ed11f24a3fb3642b9c258c90e4f"}
Jul 09 14:24:27.477150 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:27.477080    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"
Jul 09 14:24:27.502599 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:27.502541    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz" podStartSLOduration=2.102748649 podStartE2EDuration="2.502523805s" totalImagesPullingTime="399.775156ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:24:27.501284314 +0000 UTC m=+1699.283659250" watchObservedRunningTime="2026-07-09 14:24:27.502523805 +0000 UTC m=+1699.284898737"
Jul 09 14:24:31.096001 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.095912    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"]
Jul 09 14:24:31.096370 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.096137    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz" podUID="c739038f-0d74-40a0-bb6e-9d4e43319971" containerName="maas-api" containerID="cri-o://fef12659969a666ea8e4a7612694577958405609f5a1f2ac35435474a80b0237" gracePeriod=30
Jul 09 14:24:31.338212 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.338190    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"
Jul 09 14:24:31.412582 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.412556    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c739038f-0d74-40a0-bb6e-9d4e43319971-maas-api-tls\") pod \"c739038f-0d74-40a0-bb6e-9d4e43319971\" (UID: \"c739038f-0d74-40a0-bb6e-9d4e43319971\") "
Jul 09 14:24:31.412792 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.412613    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tqpm\" (UniqueName: \"kubernetes.io/projected/c739038f-0d74-40a0-bb6e-9d4e43319971-kube-api-access-6tqpm\") pod \"c739038f-0d74-40a0-bb6e-9d4e43319971\" (UID: \"c739038f-0d74-40a0-bb6e-9d4e43319971\") "
Jul 09 14:24:31.414827 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.414804    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c739038f-0d74-40a0-bb6e-9d4e43319971-kube-api-access-6tqpm" (OuterVolumeSpecName: "kube-api-access-6tqpm") pod "c739038f-0d74-40a0-bb6e-9d4e43319971" (UID: "c739038f-0d74-40a0-bb6e-9d4e43319971"). InnerVolumeSpecName "kube-api-access-6tqpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:24:31.414893 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.414829    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c739038f-0d74-40a0-bb6e-9d4e43319971-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c739038f-0d74-40a0-bb6e-9d4e43319971" (UID: "c739038f-0d74-40a0-bb6e-9d4e43319971"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:24:31.494859 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.494824    2579 generic.go:356] "Generic (PLEG): container finished" podID="c739038f-0d74-40a0-bb6e-9d4e43319971" containerID="fef12659969a666ea8e4a7612694577958405609f5a1f2ac35435474a80b0237" exitCode=0
Jul 09 14:24:31.495018 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.494871    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"
Jul 09 14:24:31.495018 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.494903    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz" event={"ID":"c739038f-0d74-40a0-bb6e-9d4e43319971","Type":"ContainerDied","Data":"fef12659969a666ea8e4a7612694577958405609f5a1f2ac35435474a80b0237"}
Jul 09 14:24:31.495018 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.494953    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz" event={"ID":"c739038f-0d74-40a0-bb6e-9d4e43319971","Type":"ContainerDied","Data":"d13f940439de1b67ab0010ec17f5864226e99ed11f24a3fb3642b9c258c90e4f"}
Jul 09 14:24:31.495018 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.494982    2579 scope.go:117] "RemoveContainer" containerID="fef12659969a666ea8e4a7612694577958405609f5a1f2ac35435474a80b0237"
Jul 09 14:24:31.502651 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.502637    2579 scope.go:117] "RemoveContainer" containerID="fef12659969a666ea8e4a7612694577958405609f5a1f2ac35435474a80b0237"
Jul 09 14:24:31.502929 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:24:31.502911    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fef12659969a666ea8e4a7612694577958405609f5a1f2ac35435474a80b0237\": container with ID starting with fef12659969a666ea8e4a7612694577958405609f5a1f2ac35435474a80b0237 not found: ID does not exist" containerID="fef12659969a666ea8e4a7612694577958405609f5a1f2ac35435474a80b0237"
Jul 09 14:24:31.502980 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.502938    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef12659969a666ea8e4a7612694577958405609f5a1f2ac35435474a80b0237"} err="failed to get container status \"fef12659969a666ea8e4a7612694577958405609f5a1f2ac35435474a80b0237\": rpc error: code = NotFound desc = could not find container \"fef12659969a666ea8e4a7612694577958405609f5a1f2ac35435474a80b0237\": container with ID starting with fef12659969a666ea8e4a7612694577958405609f5a1f2ac35435474a80b0237 not found: ID does not exist"
Jul 09 14:24:31.513266 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.513248    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c739038f-0d74-40a0-bb6e-9d4e43319971-openshift-service-ca\") pod \"c739038f-0d74-40a0-bb6e-9d4e43319971\" (UID: \"c739038f-0d74-40a0-bb6e-9d4e43319971\") "
Jul 09 14:24:31.513407 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.513394    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tqpm\" (UniqueName: \"kubernetes.io/projected/c739038f-0d74-40a0-bb6e-9d4e43319971-kube-api-access-6tqpm\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:24:31.513447 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.513410    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c739038f-0d74-40a0-bb6e-9d4e43319971-maas-api-tls\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:24:31.513538 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.513521    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c739038f-0d74-40a0-bb6e-9d4e43319971-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c739038f-0d74-40a0-bb6e-9d4e43319971" (UID: "c739038f-0d74-40a0-bb6e-9d4e43319971"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:24:31.614094 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.614063    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c739038f-0d74-40a0-bb6e-9d4e43319971-openshift-service-ca\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:24:31.820228 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.820202    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"]
Jul 09 14:24:31.826292 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:31.826267    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f4ec3dfc-54dcd96cf7-5vwnz"]
Jul 09 14:24:32.824115 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:32.824081    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c739038f-0d74-40a0-bb6e-9d4e43319971" path="/var/lib/kubelet/pods/c739038f-0d74-40a0-bb6e-9d4e43319971/volumes"
Jul 09 14:24:44.739797 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:44.739738    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"]
Jul 09 14:24:44.740254 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:44.740180    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c739038f-0d74-40a0-bb6e-9d4e43319971" containerName="maas-api"
Jul 09 14:24:44.740254 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:44.740197    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c739038f-0d74-40a0-bb6e-9d4e43319971" containerName="maas-api"
Jul 09 14:24:44.740335 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:44.740265    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c739038f-0d74-40a0-bb6e-9d4e43319971" containerName="maas-api"
Jul 09 14:24:44.743165 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:44.743148    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"
Jul 09 14:24:44.746951 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:44.746928    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-6c569375-serving-cert\""
Jul 09 14:24:44.759037 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:44.759014    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"]
Jul 09 14:24:44.820455 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:44.820421    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zlwg\" (UniqueName: \"kubernetes.io/projected/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-kube-api-access-6zlwg\") pod \"maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt\" (UID: \"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"
Jul 09 14:24:44.820573 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:44.820459    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-openshift-service-ca\") pod \"maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt\" (UID: \"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"
Jul 09 14:24:44.820573 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:44.820485    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-maas-api-tls\") pod \"maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt\" (UID: \"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"
Jul 09 14:24:44.920978 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:44.920944    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-maas-api-tls\") pod \"maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt\" (UID: \"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"
Jul 09 14:24:44.921143 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:44.921018    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zlwg\" (UniqueName: \"kubernetes.io/projected/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-kube-api-access-6zlwg\") pod \"maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt\" (UID: \"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"
Jul 09 14:24:44.921143 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:44.921044    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-openshift-service-ca\") pod \"maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt\" (UID: \"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"
Jul 09 14:24:44.921143 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:24:44.921102    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-serving-cert: secret "maas-api-e2e-derive-6c569375-serving-cert" not found
Jul 09 14:24:44.921307 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:24:44.921189    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-maas-api-tls podName:6e0da425-f82c-4c33-9cbf-251ef6ed4cb7 nodeName:}" failed. No retries permitted until 2026-07-09 14:24:45.421168393 +0000 UTC m=+1717.203543311 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-maas-api-tls") pod "maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt" (UID: "6e0da425-f82c-4c33-9cbf-251ef6ed4cb7") : secret "maas-api-e2e-derive-6c569375-serving-cert" not found
Jul 09 14:24:44.921580 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:44.921562    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-openshift-service-ca\") pod \"maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt\" (UID: \"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"
Jul 09 14:24:44.950468 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:44.950440    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zlwg\" (UniqueName: \"kubernetes.io/projected/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-kube-api-access-6zlwg\") pod \"maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt\" (UID: \"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"
Jul 09 14:24:45.425507 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:45.425474    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-maas-api-tls\") pod \"maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt\" (UID: \"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"
Jul 09 14:24:45.427976 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:45.427948    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-maas-api-tls\") pod \"maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt\" (UID: \"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"
Jul 09 14:24:45.653791 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:45.653762    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"
Jul 09 14:24:45.797137 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:45.797110    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"]
Jul 09 14:24:45.799114 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:24:45.799088    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e0da425_f82c_4c33_9cbf_251ef6ed4cb7.slice/crio-c95ae54f1023260a89783bbd29268357d7d6fed9e4047251146e5b9a886c124c WatchSource:0}: Error finding container c95ae54f1023260a89783bbd29268357d7d6fed9e4047251146e5b9a886c124c: Status 404 returned error can't find the container with id c95ae54f1023260a89783bbd29268357d7d6fed9e4047251146e5b9a886c124c
Jul 09 14:24:46.560680 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:46.560585    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt" event={"ID":"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7","Type":"ContainerStarted","Data":"54630290d3e1ccc81710a85b856709d77bd3e40fe9f296808cb8b3c496cd3fb9"}
Jul 09 14:24:46.560680 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:46.560639    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt" event={"ID":"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7","Type":"ContainerStarted","Data":"c95ae54f1023260a89783bbd29268357d7d6fed9e4047251146e5b9a886c124c"}
Jul 09 14:24:46.560905 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:46.560730    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"
Jul 09 14:24:46.583470 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:46.583413    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt" podStartSLOduration=2.12962326 podStartE2EDuration="2.583396437s" totalImagesPullingTime="453.773177ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:24:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:24:46.582307613 +0000 UTC m=+1718.364682550" watchObservedRunningTime="2026-07-09 14:24:46.583396437 +0000 UTC m=+1718.365771373"
Jul 09 14:24:50.216812 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.216778    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"]
Jul 09 14:24:50.217322 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.216984    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt" podUID="6e0da425-f82c-4c33-9cbf-251ef6ed4cb7" containerName="maas-api" containerID="cri-o://54630290d3e1ccc81710a85b856709d77bd3e40fe9f296808cb8b3c496cd3fb9" gracePeriod=30
Jul 09 14:24:50.459430 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.459407    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"
Jul 09 14:24:50.571666 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.571572    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-openshift-service-ca\") pod \"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7\" (UID: \"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7\") "
Jul 09 14:24:50.571666 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.571630    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-maas-api-tls\") pod \"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7\" (UID: \"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7\") "
Jul 09 14:24:50.571666 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.571663    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zlwg\" (UniqueName: \"kubernetes.io/projected/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-kube-api-access-6zlwg\") pod \"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7\" (UID: \"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7\") "
Jul 09 14:24:50.571990 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.571968    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6e0da425-f82c-4c33-9cbf-251ef6ed4cb7" (UID: "6e0da425-f82c-4c33-9cbf-251ef6ed4cb7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:24:50.573937 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.573912    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-kube-api-access-6zlwg" (OuterVolumeSpecName: "kube-api-access-6zlwg") pod "6e0da425-f82c-4c33-9cbf-251ef6ed4cb7" (UID: "6e0da425-f82c-4c33-9cbf-251ef6ed4cb7"). InnerVolumeSpecName "kube-api-access-6zlwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:24:50.574058 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.573941    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6e0da425-f82c-4c33-9cbf-251ef6ed4cb7" (UID: "6e0da425-f82c-4c33-9cbf-251ef6ed4cb7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:24:50.578918 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.578854    2579 generic.go:356] "Generic (PLEG): container finished" podID="6e0da425-f82c-4c33-9cbf-251ef6ed4cb7" containerID="54630290d3e1ccc81710a85b856709d77bd3e40fe9f296808cb8b3c496cd3fb9" exitCode=0
Jul 09 14:24:50.578918 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.578897    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"
Jul 09 14:24:50.578994 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.578934    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt" event={"ID":"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7","Type":"ContainerDied","Data":"54630290d3e1ccc81710a85b856709d77bd3e40fe9f296808cb8b3c496cd3fb9"}
Jul 09 14:24:50.579028 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.578995    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt" event={"ID":"6e0da425-f82c-4c33-9cbf-251ef6ed4cb7","Type":"ContainerDied","Data":"c95ae54f1023260a89783bbd29268357d7d6fed9e4047251146e5b9a886c124c"}
Jul 09 14:24:50.579028 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.579024    2579 scope.go:117] "RemoveContainer" containerID="54630290d3e1ccc81710a85b856709d77bd3e40fe9f296808cb8b3c496cd3fb9"
Jul 09 14:24:50.587300 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.587285    2579 scope.go:117] "RemoveContainer" containerID="54630290d3e1ccc81710a85b856709d77bd3e40fe9f296808cb8b3c496cd3fb9"
Jul 09 14:24:50.587544 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:24:50.587521    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54630290d3e1ccc81710a85b856709d77bd3e40fe9f296808cb8b3c496cd3fb9\": container with ID starting with 54630290d3e1ccc81710a85b856709d77bd3e40fe9f296808cb8b3c496cd3fb9 not found: ID does not exist" containerID="54630290d3e1ccc81710a85b856709d77bd3e40fe9f296808cb8b3c496cd3fb9"
Jul 09 14:24:50.587602 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.587553    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54630290d3e1ccc81710a85b856709d77bd3e40fe9f296808cb8b3c496cd3fb9"} err="failed to get container status \"54630290d3e1ccc81710a85b856709d77bd3e40fe9f296808cb8b3c496cd3fb9\": rpc error: code = NotFound desc = could not find container \"54630290d3e1ccc81710a85b856709d77bd3e40fe9f296808cb8b3c496cd3fb9\": container with ID starting with 54630290d3e1ccc81710a85b856709d77bd3e40fe9f296808cb8b3c496cd3fb9 not found: ID does not exist"
Jul 09 14:24:50.601923 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.601902    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"]
Jul 09 14:24:50.605987 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.605969    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c569375-6c99b78f5d-xcmwt"]
Jul 09 14:24:50.672954 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.672920    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zlwg\" (UniqueName: \"kubernetes.io/projected/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-kube-api-access-6zlwg\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:24:50.672954 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.672949    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-openshift-service-ca\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:24:50.672954 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.672962    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7-maas-api-tls\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:24:50.824742 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:24:50.824659    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e0da425-f82c-4c33-9cbf-251ef6ed4cb7" path="/var/lib/kubelet/pods/6e0da425-f82c-4c33-9cbf-251ef6ed4cb7/volumes"
Jul 09 14:26:08.770535 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:08.770507    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 14:26:08.771551 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:08.771527    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 14:26:38.710439 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:38.710405    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"]
Jul 09 14:26:38.710909 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:38.710702    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e0da425-f82c-4c33-9cbf-251ef6ed4cb7" containerName="maas-api"
Jul 09 14:26:38.710909 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:38.710712    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e0da425-f82c-4c33-9cbf-251ef6ed4cb7" containerName="maas-api"
Jul 09 14:26:38.710909 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:38.710803    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e0da425-f82c-4c33-9cbf-251ef6ed4cb7" containerName="maas-api"
Jul 09 14:26:38.713611 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:38.713594    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"
Jul 09 14:26:38.717183 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:38.717160    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9c958fc8-serving-cert\""
Jul 09 14:26:38.737090 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:38.737064    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"]
Jul 09 14:26:38.813519 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:38.813483    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccc1da6d-2509-4973-8dac-989f52ee90d1-openshift-service-ca\") pod \"maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl\" (UID: \"ccc1da6d-2509-4973-8dac-989f52ee90d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"
Jul 09 14:26:38.813730 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:38.813530    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccc1da6d-2509-4973-8dac-989f52ee90d1-maas-api-tls\") pod \"maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl\" (UID: \"ccc1da6d-2509-4973-8dac-989f52ee90d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"
Jul 09 14:26:38.813730 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:38.813552    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84b5z\" (UniqueName: \"kubernetes.io/projected/ccc1da6d-2509-4973-8dac-989f52ee90d1-kube-api-access-84b5z\") pod \"maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl\" (UID: \"ccc1da6d-2509-4973-8dac-989f52ee90d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"
Jul 09 14:26:38.914607 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:38.914561    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84b5z\" (UniqueName: \"kubernetes.io/projected/ccc1da6d-2509-4973-8dac-989f52ee90d1-kube-api-access-84b5z\") pod \"maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl\" (UID: \"ccc1da6d-2509-4973-8dac-989f52ee90d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"
Jul 09 14:26:38.914868 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:38.914672    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccc1da6d-2509-4973-8dac-989f52ee90d1-openshift-service-ca\") pod \"maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl\" (UID: \"ccc1da6d-2509-4973-8dac-989f52ee90d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"
Jul 09 14:26:38.914868 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:38.914702    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccc1da6d-2509-4973-8dac-989f52ee90d1-maas-api-tls\") pod \"maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl\" (UID: \"ccc1da6d-2509-4973-8dac-989f52ee90d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"
Jul 09 14:26:38.914868 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:26:38.914823    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-serving-cert: secret "maas-api-e2e-mt-9c958fc8-serving-cert" not found
Jul 09 14:26:38.915024 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:26:38.914882    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccc1da6d-2509-4973-8dac-989f52ee90d1-maas-api-tls podName:ccc1da6d-2509-4973-8dac-989f52ee90d1 nodeName:}" failed. No retries permitted until 2026-07-09 14:26:39.414863597 +0000 UTC m=+1831.197238529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ccc1da6d-2509-4973-8dac-989f52ee90d1-maas-api-tls") pod "maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl" (UID: "ccc1da6d-2509-4973-8dac-989f52ee90d1") : secret "maas-api-e2e-mt-9c958fc8-serving-cert" not found
Jul 09 14:26:38.915321 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:38.915300    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccc1da6d-2509-4973-8dac-989f52ee90d1-openshift-service-ca\") pod \"maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl\" (UID: \"ccc1da6d-2509-4973-8dac-989f52ee90d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"
Jul 09 14:26:38.925373 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:38.925348    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84b5z\" (UniqueName: \"kubernetes.io/projected/ccc1da6d-2509-4973-8dac-989f52ee90d1-kube-api-access-84b5z\") pod \"maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl\" (UID: \"ccc1da6d-2509-4973-8dac-989f52ee90d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"
Jul 09 14:26:39.420071 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:39.420032    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccc1da6d-2509-4973-8dac-989f52ee90d1-maas-api-tls\") pod \"maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl\" (UID: \"ccc1da6d-2509-4973-8dac-989f52ee90d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"
Jul 09 14:26:39.422589 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:39.422565    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccc1da6d-2509-4973-8dac-989f52ee90d1-maas-api-tls\") pod \"maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl\" (UID: \"ccc1da6d-2509-4973-8dac-989f52ee90d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"
Jul 09 14:26:39.622856 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:39.622822    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"
Jul 09 14:26:39.801907 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:26:39.801878    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccc1da6d_2509_4973_8dac_989f52ee90d1.slice/crio-f2cbc7c39221461106d3253af84d5a2984a92beb3a084e359cdf6da576b2597c WatchSource:0}: Error finding container f2cbc7c39221461106d3253af84d5a2984a92beb3a084e359cdf6da576b2597c: Status 404 returned error can't find the container with id f2cbc7c39221461106d3253af84d5a2984a92beb3a084e359cdf6da576b2597c
Jul 09 14:26:39.828935 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:39.826338    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"]
Jul 09 14:26:40.036817 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:40.036717    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl" event={"ID":"ccc1da6d-2509-4973-8dac-989f52ee90d1","Type":"ContainerStarted","Data":"f2cbc7c39221461106d3253af84d5a2984a92beb3a084e359cdf6da576b2597c"}
Jul 09 14:26:41.041640 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:41.041602    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl" event={"ID":"ccc1da6d-2509-4973-8dac-989f52ee90d1","Type":"ContainerStarted","Data":"82fb98a46b57b39ad9eeb951eb4e6961169ea79456f398a9fab1688a7cf47f90"}
Jul 09 14:26:41.042049 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:41.041734    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"
Jul 09 14:26:41.061476 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:41.061430    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl" podStartSLOduration=2.626646364 podStartE2EDuration="3.061414176s" totalImagesPullingTime="434.767812ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:26:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:26:41.059387068 +0000 UTC m=+1832.841762018" watchObservedRunningTime="2026-07-09 14:26:41.061414176 +0000 UTC m=+1832.843789111"
Jul 09 14:26:47.050963 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:47.050935    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"
Jul 09 14:26:51.253317 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:51.253285    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"]
Jul 09 14:26:51.253792 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:51.253518    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl" podUID="ccc1da6d-2509-4973-8dac-989f52ee90d1" containerName="maas-api" containerID="cri-o://82fb98a46b57b39ad9eeb951eb4e6961169ea79456f398a9fab1688a7cf47f90" gracePeriod=30
Jul 09 14:26:51.491051 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:51.491027    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"
Jul 09 14:26:51.628850 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:51.628766    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccc1da6d-2509-4973-8dac-989f52ee90d1-maas-api-tls\") pod \"ccc1da6d-2509-4973-8dac-989f52ee90d1\" (UID: \"ccc1da6d-2509-4973-8dac-989f52ee90d1\") "
Jul 09 14:26:51.628850 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:51.628823    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84b5z\" (UniqueName: \"kubernetes.io/projected/ccc1da6d-2509-4973-8dac-989f52ee90d1-kube-api-access-84b5z\") pod \"ccc1da6d-2509-4973-8dac-989f52ee90d1\" (UID: \"ccc1da6d-2509-4973-8dac-989f52ee90d1\") "
Jul 09 14:26:51.629027 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:51.628866    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccc1da6d-2509-4973-8dac-989f52ee90d1-openshift-service-ca\") pod \"ccc1da6d-2509-4973-8dac-989f52ee90d1\" (UID: \"ccc1da6d-2509-4973-8dac-989f52ee90d1\") "
Jul 09 14:26:51.629237 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:51.629212    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccc1da6d-2509-4973-8dac-989f52ee90d1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ccc1da6d-2509-4973-8dac-989f52ee90d1" (UID: "ccc1da6d-2509-4973-8dac-989f52ee90d1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:26:51.630815 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:51.630790    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccc1da6d-2509-4973-8dac-989f52ee90d1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ccc1da6d-2509-4973-8dac-989f52ee90d1" (UID: "ccc1da6d-2509-4973-8dac-989f52ee90d1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:26:51.631058 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:51.631031    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccc1da6d-2509-4973-8dac-989f52ee90d1-kube-api-access-84b5z" (OuterVolumeSpecName: "kube-api-access-84b5z") pod "ccc1da6d-2509-4973-8dac-989f52ee90d1" (UID: "ccc1da6d-2509-4973-8dac-989f52ee90d1"). InnerVolumeSpecName "kube-api-access-84b5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:26:51.729896 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:51.729858    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccc1da6d-2509-4973-8dac-989f52ee90d1-openshift-service-ca\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:26:51.729896 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:51.729890    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccc1da6d-2509-4973-8dac-989f52ee90d1-maas-api-tls\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:26:51.729896 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:51.729903    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84b5z\" (UniqueName: \"kubernetes.io/projected/ccc1da6d-2509-4973-8dac-989f52ee90d1-kube-api-access-84b5z\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:26:52.090522 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:52.090481    2579 generic.go:356] "Generic (PLEG): container finished" podID="ccc1da6d-2509-4973-8dac-989f52ee90d1" containerID="82fb98a46b57b39ad9eeb951eb4e6961169ea79456f398a9fab1688a7cf47f90" exitCode=0
Jul 09 14:26:52.090522 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:52.090525    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"
Jul 09 14:26:52.090777 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:52.090559    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl" event={"ID":"ccc1da6d-2509-4973-8dac-989f52ee90d1","Type":"ContainerDied","Data":"82fb98a46b57b39ad9eeb951eb4e6961169ea79456f398a9fab1688a7cf47f90"}
Jul 09 14:26:52.090777 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:52.090611    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl" event={"ID":"ccc1da6d-2509-4973-8dac-989f52ee90d1","Type":"ContainerDied","Data":"f2cbc7c39221461106d3253af84d5a2984a92beb3a084e359cdf6da576b2597c"}
Jul 09 14:26:52.090777 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:52.090637    2579 scope.go:117] "RemoveContainer" containerID="82fb98a46b57b39ad9eeb951eb4e6961169ea79456f398a9fab1688a7cf47f90"
Jul 09 14:26:52.099632 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:52.099614    2579 scope.go:117] "RemoveContainer" containerID="82fb98a46b57b39ad9eeb951eb4e6961169ea79456f398a9fab1688a7cf47f90"
Jul 09 14:26:52.099895 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:26:52.099875    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82fb98a46b57b39ad9eeb951eb4e6961169ea79456f398a9fab1688a7cf47f90\": container with ID starting with 82fb98a46b57b39ad9eeb951eb4e6961169ea79456f398a9fab1688a7cf47f90 not found: ID does not exist" containerID="82fb98a46b57b39ad9eeb951eb4e6961169ea79456f398a9fab1688a7cf47f90"
Jul 09 14:26:52.099949 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:52.099905    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82fb98a46b57b39ad9eeb951eb4e6961169ea79456f398a9fab1688a7cf47f90"} err="failed to get container status \"82fb98a46b57b39ad9eeb951eb4e6961169ea79456f398a9fab1688a7cf47f90\": rpc error: code = NotFound desc = could not find container \"82fb98a46b57b39ad9eeb951eb4e6961169ea79456f398a9fab1688a7cf47f90\": container with ID starting with 82fb98a46b57b39ad9eeb951eb4e6961169ea79456f398a9fab1688a7cf47f90 not found: ID does not exist"
Jul 09 14:26:52.118356 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:52.118324    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"]
Jul 09 14:26:52.121343 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:52.121322    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9c958fc8-5b59f47cd5-m9rnl"]
Jul 09 14:26:52.825035 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:26:52.825003    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccc1da6d-2509-4973-8dac-989f52ee90d1" path="/var/lib/kubelet/pods/ccc1da6d-2509-4973-8dac-989f52ee90d1/volumes"
Jul 09 14:27:20.705173 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:20.705133    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"]
Jul 09 14:27:20.705726 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:20.705446    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccc1da6d-2509-4973-8dac-989f52ee90d1" containerName="maas-api"
Jul 09 14:27:20.705726 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:20.705457    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccc1da6d-2509-4973-8dac-989f52ee90d1" containerName="maas-api"
Jul 09 14:27:20.705726 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:20.705506    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccc1da6d-2509-4973-8dac-989f52ee90d1" containerName="maas-api"
Jul 09 14:27:20.708494 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:20.708475    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"
Jul 09 14:27:20.712111 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:20.712087    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-bea9ff30-serving-cert\""
Jul 09 14:27:20.730098 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:20.730075    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"]
Jul 09 14:27:20.784941 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:20.784903    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/971983bc-1a93-469b-bf88-9996bceb9238-openshift-service-ca\") pod \"maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5\" (UID: \"971983bc-1a93-469b-bf88-9996bceb9238\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"
Jul 09 14:27:20.785131 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:20.785044    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwrfm\" (UniqueName: \"kubernetes.io/projected/971983bc-1a93-469b-bf88-9996bceb9238-kube-api-access-xwrfm\") pod \"maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5\" (UID: \"971983bc-1a93-469b-bf88-9996bceb9238\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"
Jul 09 14:27:20.785131 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:20.785075    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/971983bc-1a93-469b-bf88-9996bceb9238-maas-api-tls\") pod \"maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5\" (UID: \"971983bc-1a93-469b-bf88-9996bceb9238\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"
Jul 09 14:27:20.885664 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:20.885628    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/971983bc-1a93-469b-bf88-9996bceb9238-maas-api-tls\") pod \"maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5\" (UID: \"971983bc-1a93-469b-bf88-9996bceb9238\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"
Jul 09 14:27:20.885880 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:20.885681    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/971983bc-1a93-469b-bf88-9996bceb9238-openshift-service-ca\") pod \"maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5\" (UID: \"971983bc-1a93-469b-bf88-9996bceb9238\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"
Jul 09 14:27:20.885880 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:20.885788    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwrfm\" (UniqueName: \"kubernetes.io/projected/971983bc-1a93-469b-bf88-9996bceb9238-kube-api-access-xwrfm\") pod \"maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5\" (UID: \"971983bc-1a93-469b-bf88-9996bceb9238\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"
Jul 09 14:27:20.885880 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:27:20.885831    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-serving-cert: secret "maas-api-e2e-mt-bea9ff30-serving-cert" not found
Jul 09 14:27:20.886050 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:27:20.885909    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971983bc-1a93-469b-bf88-9996bceb9238-maas-api-tls podName:971983bc-1a93-469b-bf88-9996bceb9238 nodeName:}" failed. No retries permitted until 2026-07-09 14:27:21.38588857 +0000 UTC m=+1873.168263497 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/971983bc-1a93-469b-bf88-9996bceb9238-maas-api-tls") pod "maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5" (UID: "971983bc-1a93-469b-bf88-9996bceb9238") : secret "maas-api-e2e-mt-bea9ff30-serving-cert" not found
Jul 09 14:27:20.886361 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:20.886340    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/971983bc-1a93-469b-bf88-9996bceb9238-openshift-service-ca\") pod \"maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5\" (UID: \"971983bc-1a93-469b-bf88-9996bceb9238\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"
Jul 09 14:27:20.906154 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:20.906129    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwrfm\" (UniqueName: \"kubernetes.io/projected/971983bc-1a93-469b-bf88-9996bceb9238-kube-api-access-xwrfm\") pod \"maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5\" (UID: \"971983bc-1a93-469b-bf88-9996bceb9238\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"
Jul 09 14:27:21.389799 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:21.389741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/971983bc-1a93-469b-bf88-9996bceb9238-maas-api-tls\") pod \"maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5\" (UID: \"971983bc-1a93-469b-bf88-9996bceb9238\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"
Jul 09 14:27:21.392264 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:21.392245    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/971983bc-1a93-469b-bf88-9996bceb9238-maas-api-tls\") pod \"maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5\" (UID: \"971983bc-1a93-469b-bf88-9996bceb9238\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"
Jul 09 14:27:21.619156 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:21.619125    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"
Jul 09 14:27:21.755282 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:21.755257    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"]
Jul 09 14:27:21.757936 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:27:21.757898    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod971983bc_1a93_469b_bf88_9996bceb9238.slice/crio-51970ab2f9ba3f2620827ff0ee62c2b0df9de6925faa04168142e0a23a533615 WatchSource:0}: Error finding container 51970ab2f9ba3f2620827ff0ee62c2b0df9de6925faa04168142e0a23a533615: Status 404 returned error can't find the container with id 51970ab2f9ba3f2620827ff0ee62c2b0df9de6925faa04168142e0a23a533615
Jul 09 14:27:22.219014 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:22.218978    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5" event={"ID":"971983bc-1a93-469b-bf88-9996bceb9238","Type":"ContainerStarted","Data":"51970ab2f9ba3f2620827ff0ee62c2b0df9de6925faa04168142e0a23a533615"}
Jul 09 14:27:23.229093 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:23.229050    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5" event={"ID":"971983bc-1a93-469b-bf88-9996bceb9238","Type":"ContainerStarted","Data":"509d5a39508eedc6e045492957aaa3d98c5ef34614b542202b1a1cb4aad0619a"}
Jul 09 14:27:23.229477 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:23.229163    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"
Jul 09 14:27:23.248611 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:23.248558    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5" podStartSLOduration=2.257928176 podStartE2EDuration="3.248542921s" totalImagesPullingTime="990.614745ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:27:23.247813473 +0000 UTC m=+1875.030188408" watchObservedRunningTime="2026-07-09 14:27:23.248542921 +0000 UTC m=+1875.030917886"
Jul 09 14:27:29.239236 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:27:29.239155    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"
Jul 09 14:28:08.322331 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:08.322300    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"]
Jul 09 14:28:08.322864 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:08.322536    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5" podUID="971983bc-1a93-469b-bf88-9996bceb9238" containerName="maas-api" containerID="cri-o://509d5a39508eedc6e045492957aaa3d98c5ef34614b542202b1a1cb4aad0619a" gracePeriod=30
Jul 09 14:28:08.566483 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:08.566452    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"
Jul 09 14:28:08.694835 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:08.694794    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/971983bc-1a93-469b-bf88-9996bceb9238-maas-api-tls\") pod \"971983bc-1a93-469b-bf88-9996bceb9238\" (UID: \"971983bc-1a93-469b-bf88-9996bceb9238\") "
Jul 09 14:28:08.694835 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:08.694838    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwrfm\" (UniqueName: \"kubernetes.io/projected/971983bc-1a93-469b-bf88-9996bceb9238-kube-api-access-xwrfm\") pod \"971983bc-1a93-469b-bf88-9996bceb9238\" (UID: \"971983bc-1a93-469b-bf88-9996bceb9238\") "
Jul 09 14:28:08.695093 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:08.694859    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/971983bc-1a93-469b-bf88-9996bceb9238-openshift-service-ca\") pod \"971983bc-1a93-469b-bf88-9996bceb9238\" (UID: \"971983bc-1a93-469b-bf88-9996bceb9238\") "
Jul 09 14:28:08.695281 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:08.695254    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/971983bc-1a93-469b-bf88-9996bceb9238-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "971983bc-1a93-469b-bf88-9996bceb9238" (UID: "971983bc-1a93-469b-bf88-9996bceb9238"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:28:08.697141 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:08.697118    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/971983bc-1a93-469b-bf88-9996bceb9238-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "971983bc-1a93-469b-bf88-9996bceb9238" (UID: "971983bc-1a93-469b-bf88-9996bceb9238"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:28:08.697211 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:08.697131    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/971983bc-1a93-469b-bf88-9996bceb9238-kube-api-access-xwrfm" (OuterVolumeSpecName: "kube-api-access-xwrfm") pod "971983bc-1a93-469b-bf88-9996bceb9238" (UID: "971983bc-1a93-469b-bf88-9996bceb9238"). InnerVolumeSpecName "kube-api-access-xwrfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:28:08.795615 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:08.795570    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/971983bc-1a93-469b-bf88-9996bceb9238-maas-api-tls\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:28:08.795615 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:08.795604    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwrfm\" (UniqueName: \"kubernetes.io/projected/971983bc-1a93-469b-bf88-9996bceb9238-kube-api-access-xwrfm\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:28:08.795615 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:08.795613    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/971983bc-1a93-469b-bf88-9996bceb9238-openshift-service-ca\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:28:09.425806 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:09.425768    2579 generic.go:356] "Generic (PLEG): container finished" podID="971983bc-1a93-469b-bf88-9996bceb9238" containerID="509d5a39508eedc6e045492957aaa3d98c5ef34614b542202b1a1cb4aad0619a" exitCode=0
Jul 09 14:28:09.426203 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:09.425838    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"
Jul 09 14:28:09.426203 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:09.425839    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5" event={"ID":"971983bc-1a93-469b-bf88-9996bceb9238","Type":"ContainerDied","Data":"509d5a39508eedc6e045492957aaa3d98c5ef34614b542202b1a1cb4aad0619a"}
Jul 09 14:28:09.426203 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:09.425952    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5" event={"ID":"971983bc-1a93-469b-bf88-9996bceb9238","Type":"ContainerDied","Data":"51970ab2f9ba3f2620827ff0ee62c2b0df9de6925faa04168142e0a23a533615"}
Jul 09 14:28:09.426203 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:09.425972    2579 scope.go:117] "RemoveContainer" containerID="509d5a39508eedc6e045492957aaa3d98c5ef34614b542202b1a1cb4aad0619a"
Jul 09 14:28:09.434197 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:09.434174    2579 scope.go:117] "RemoveContainer" containerID="509d5a39508eedc6e045492957aaa3d98c5ef34614b542202b1a1cb4aad0619a"
Jul 09 14:28:09.434486 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:28:09.434463    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"509d5a39508eedc6e045492957aaa3d98c5ef34614b542202b1a1cb4aad0619a\": container with ID starting with 509d5a39508eedc6e045492957aaa3d98c5ef34614b542202b1a1cb4aad0619a not found: ID does not exist" containerID="509d5a39508eedc6e045492957aaa3d98c5ef34614b542202b1a1cb4aad0619a"
Jul 09 14:28:09.434555 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:09.434494    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"509d5a39508eedc6e045492957aaa3d98c5ef34614b542202b1a1cb4aad0619a"} err="failed to get container status \"509d5a39508eedc6e045492957aaa3d98c5ef34614b542202b1a1cb4aad0619a\": rpc error: code = NotFound desc = could not find container \"509d5a39508eedc6e045492957aaa3d98c5ef34614b542202b1a1cb4aad0619a\": container with ID starting with 509d5a39508eedc6e045492957aaa3d98c5ef34614b542202b1a1cb4aad0619a not found: ID does not exist"
Jul 09 14:28:09.443867 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:09.443844    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"]
Jul 09 14:28:09.447808 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:09.447786    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bea9ff30-7569956cbd-8rvg5"]
Jul 09 14:28:10.824398 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:28:10.824367    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="971983bc-1a93-469b-bf88-9996bceb9238" path="/var/lib/kubelet/pods/971983bc-1a93-469b-bf88-9996bceb9238/volumes"
Jul 09 14:30:01.940123 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:01.940089    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"]
Jul 09 14:30:01.940514 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:01.940413    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="971983bc-1a93-469b-bf88-9996bceb9238" containerName="maas-api"
Jul 09 14:30:01.940514 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:01.940424    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="971983bc-1a93-469b-bf88-9996bceb9238" containerName="maas-api"
Jul 09 14:30:01.940514 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:01.940475    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="971983bc-1a93-469b-bf88-9996bceb9238" containerName="maas-api"
Jul 09 14:30:01.943377 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:01.943359    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"
Jul 09 14:30:01.945867 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:01.945842    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-738843-serving-cert\""
Jul 09 14:30:01.956508 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:01.956481    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"]
Jul 09 14:30:02.100326 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:02.100290    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b105de2-85ef-4694-beee-8d3f40f11390-openshift-service-ca\") pod \"maas-api-e2e-inf-a-738843-89f6cd765-cj5tv\" (UID: \"8b105de2-85ef-4694-beee-8d3f40f11390\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"
Jul 09 14:30:02.100507 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:02.100341    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktc6f\" (UniqueName: \"kubernetes.io/projected/8b105de2-85ef-4694-beee-8d3f40f11390-kube-api-access-ktc6f\") pod \"maas-api-e2e-inf-a-738843-89f6cd765-cj5tv\" (UID: \"8b105de2-85ef-4694-beee-8d3f40f11390\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"
Jul 09 14:30:02.100507 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:02.100423    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b105de2-85ef-4694-beee-8d3f40f11390-maas-api-tls\") pod \"maas-api-e2e-inf-a-738843-89f6cd765-cj5tv\" (UID: \"8b105de2-85ef-4694-beee-8d3f40f11390\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"
Jul 09 14:30:02.201667 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:02.201578    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b105de2-85ef-4694-beee-8d3f40f11390-openshift-service-ca\") pod \"maas-api-e2e-inf-a-738843-89f6cd765-cj5tv\" (UID: \"8b105de2-85ef-4694-beee-8d3f40f11390\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"
Jul 09 14:30:02.201667 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:02.201639    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktc6f\" (UniqueName: \"kubernetes.io/projected/8b105de2-85ef-4694-beee-8d3f40f11390-kube-api-access-ktc6f\") pod \"maas-api-e2e-inf-a-738843-89f6cd765-cj5tv\" (UID: \"8b105de2-85ef-4694-beee-8d3f40f11390\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"
Jul 09 14:30:02.201880 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:02.201711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b105de2-85ef-4694-beee-8d3f40f11390-maas-api-tls\") pod \"maas-api-e2e-inf-a-738843-89f6cd765-cj5tv\" (UID: \"8b105de2-85ef-4694-beee-8d3f40f11390\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"
Jul 09 14:30:02.202348 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:02.202313    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b105de2-85ef-4694-beee-8d3f40f11390-openshift-service-ca\") pod \"maas-api-e2e-inf-a-738843-89f6cd765-cj5tv\" (UID: \"8b105de2-85ef-4694-beee-8d3f40f11390\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"
Jul 09 14:30:02.204400 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:02.204377    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b105de2-85ef-4694-beee-8d3f40f11390-maas-api-tls\") pod \"maas-api-e2e-inf-a-738843-89f6cd765-cj5tv\" (UID: \"8b105de2-85ef-4694-beee-8d3f40f11390\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"
Jul 09 14:30:02.211410 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:02.211387    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktc6f\" (UniqueName: \"kubernetes.io/projected/8b105de2-85ef-4694-beee-8d3f40f11390-kube-api-access-ktc6f\") pod \"maas-api-e2e-inf-a-738843-89f6cd765-cj5tv\" (UID: \"8b105de2-85ef-4694-beee-8d3f40f11390\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"
Jul 09 14:30:02.253242 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:02.253208    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"
Jul 09 14:30:02.383599 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:02.383523    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"]
Jul 09 14:30:02.385933 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:30:02.385904    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b105de2_85ef_4694_beee_8d3f40f11390.slice/crio-cf5164cf7bc4b7d9e65293e041d5bd1fcb8cc8583cbc6b093226fbd18ec97fa1 WatchSource:0}: Error finding container cf5164cf7bc4b7d9e65293e041d5bd1fcb8cc8583cbc6b093226fbd18ec97fa1: Status 404 returned error can't find the container with id cf5164cf7bc4b7d9e65293e041d5bd1fcb8cc8583cbc6b093226fbd18ec97fa1
Jul 09 14:30:02.387157 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:02.387135    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 14:30:02.897304 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:02.897265    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv" event={"ID":"8b105de2-85ef-4694-beee-8d3f40f11390","Type":"ContainerStarted","Data":"cf5164cf7bc4b7d9e65293e041d5bd1fcb8cc8583cbc6b093226fbd18ec97fa1"}
Jul 09 14:30:03.902714 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:03.902678    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv" event={"ID":"8b105de2-85ef-4694-beee-8d3f40f11390","Type":"ContainerStarted","Data":"d16f6a6deb94ba29857bd18b47595bbf3db73e110eee7074a146c85744a37a20"}
Jul 09 14:30:03.903115 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:03.902812    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"
Jul 09 14:30:03.926497 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:03.926443    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv" podStartSLOduration=1.7784158460000001 podStartE2EDuration="2.926427287s" totalImagesPullingTime="1.148011441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:30:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:30:03.924839468 +0000 UTC m=+2035.707214404" watchObservedRunningTime="2026-07-09 14:30:03.926427287 +0000 UTC m=+2035.708802275"
Jul 09 14:30:09.913100 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:09.913062    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"
Jul 09 14:30:11.159690 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:11.159654    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"]
Jul 09 14:30:11.164325 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:11.164290    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:30:11.167336 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:11.167312    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-f1a2e7-serving-cert\""
Jul 09 14:30:11.178452 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:11.178429    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a58756f-fc4c-41cd-81f0-6276644415af-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj\" (UID: \"6a58756f-fc4c-41cd-81f0-6276644415af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:30:11.178597 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:11.178480    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a58756f-fc4c-41cd-81f0-6276644415af-maas-api-tls\") pod \"maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj\" (UID: \"6a58756f-fc4c-41cd-81f0-6276644415af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:30:11.178597 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:11.178505    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx65w\" (UniqueName: \"kubernetes.io/projected/6a58756f-fc4c-41cd-81f0-6276644415af-kube-api-access-hx65w\") pod \"maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj\" (UID: \"6a58756f-fc4c-41cd-81f0-6276644415af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:30:11.179836 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:11.179811    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"]
Jul 09 14:30:11.279002 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:11.278954    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a58756f-fc4c-41cd-81f0-6276644415af-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj\" (UID: \"6a58756f-fc4c-41cd-81f0-6276644415af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:30:11.279002 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:11.279016    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a58756f-fc4c-41cd-81f0-6276644415af-maas-api-tls\") pod \"maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj\" (UID: \"6a58756f-fc4c-41cd-81f0-6276644415af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:30:11.279202 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:11.279033    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx65w\" (UniqueName: \"kubernetes.io/projected/6a58756f-fc4c-41cd-81f0-6276644415af-kube-api-access-hx65w\") pod \"maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj\" (UID: \"6a58756f-fc4c-41cd-81f0-6276644415af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:30:11.279202 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:30:11.279152    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-serving-cert: secret "maas-api-e2e-inf-b-f1a2e7-serving-cert" not found
Jul 09 14:30:11.279266 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:30:11.279220    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a58756f-fc4c-41cd-81f0-6276644415af-maas-api-tls podName:6a58756f-fc4c-41cd-81f0-6276644415af nodeName:}" failed. No retries permitted until 2026-07-09 14:30:11.7792036 +0000 UTC m=+2043.561578513 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6a58756f-fc4c-41cd-81f0-6276644415af-maas-api-tls") pod "maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj" (UID: "6a58756f-fc4c-41cd-81f0-6276644415af") : secret "maas-api-e2e-inf-b-f1a2e7-serving-cert" not found
Jul 09 14:30:11.279649 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:11.279630    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a58756f-fc4c-41cd-81f0-6276644415af-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj\" (UID: \"6a58756f-fc4c-41cd-81f0-6276644415af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:30:11.288403 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:11.288379    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx65w\" (UniqueName: \"kubernetes.io/projected/6a58756f-fc4c-41cd-81f0-6276644415af-kube-api-access-hx65w\") pod \"maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj\" (UID: \"6a58756f-fc4c-41cd-81f0-6276644415af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:30:11.782484 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:11.782433    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a58756f-fc4c-41cd-81f0-6276644415af-maas-api-tls\") pod \"maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj\" (UID: \"6a58756f-fc4c-41cd-81f0-6276644415af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:30:11.782677 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:30:11.782543    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-serving-cert: secret "maas-api-e2e-inf-b-f1a2e7-serving-cert" not found
Jul 09 14:30:11.782677 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:30:11.782606    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a58756f-fc4c-41cd-81f0-6276644415af-maas-api-tls podName:6a58756f-fc4c-41cd-81f0-6276644415af nodeName:}" failed. No retries permitted until 2026-07-09 14:30:12.782590672 +0000 UTC m=+2044.564965588 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6a58756f-fc4c-41cd-81f0-6276644415af-maas-api-tls") pod "maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj" (UID: "6a58756f-fc4c-41cd-81f0-6276644415af") : secret "maas-api-e2e-inf-b-f1a2e7-serving-cert" not found
Jul 09 14:30:12.789873 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:12.789839    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a58756f-fc4c-41cd-81f0-6276644415af-maas-api-tls\") pod \"maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj\" (UID: \"6a58756f-fc4c-41cd-81f0-6276644415af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:30:12.792431 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:12.792403    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a58756f-fc4c-41cd-81f0-6276644415af-maas-api-tls\") pod \"maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj\" (UID: \"6a58756f-fc4c-41cd-81f0-6276644415af\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:30:12.974626 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:12.974578    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:30:13.110885 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:13.110686    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"]
Jul 09 14:30:13.112864 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:30:13.112832    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a58756f_fc4c_41cd_81f0_6276644415af.slice/crio-5af386a5a4beb0ced5091a6e7fb70037a67a89beed8256a1b1861e00bb6273a5 WatchSource:0}: Error finding container 5af386a5a4beb0ced5091a6e7fb70037a67a89beed8256a1b1861e00bb6273a5: Status 404 returned error can't find the container with id 5af386a5a4beb0ced5091a6e7fb70037a67a89beed8256a1b1861e00bb6273a5
Jul 09 14:30:13.946526 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:13.946482    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj" event={"ID":"6a58756f-fc4c-41cd-81f0-6276644415af","Type":"ContainerStarted","Data":"29491b8ff3fc46a4891776abd308a7be17b875516728915d9317d6f3b418546a"}
Jul 09 14:30:13.946925 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:13.946548    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj" event={"ID":"6a58756f-fc4c-41cd-81f0-6276644415af","Type":"ContainerStarted","Data":"5af386a5a4beb0ced5091a6e7fb70037a67a89beed8256a1b1861e00bb6273a5"}
Jul 09 14:30:13.946925 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:13.946575    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:30:13.966719 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:13.966672    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj" podStartSLOduration=2.627891821 podStartE2EDuration="2.966656154s" totalImagesPullingTime="338.764333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:30:13.965896902 +0000 UTC m=+2045.748271843" watchObservedRunningTime="2026-07-09 14:30:13.966656154 +0000 UTC m=+2045.749031086"
Jul 09 14:30:19.957041 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:19.957011    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:30:25.199930 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.199887    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"]
Jul 09 14:30:25.203560 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.203539    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.208125 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.208108    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-738843\"/\"kube-root-ca.crt\""
Jul 09 14:30:25.209106 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.209086    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-738843\"/\"default-dockercfg-l9vs8\""
Jul 09 14:30:25.212649 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.212631    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-738843\"/\"openshift-service-ca.crt\""
Jul 09 14:30:25.218937 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.218917    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-738843\"/\"test-model-738843-kserve-self-signed-certs\""
Jul 09 14:30:25.238922 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.238893    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"]
Jul 09 14:30:25.286666 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.286634    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-model-cache\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.286855 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.286697    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-tmp-dir\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.286855 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.286831    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-dshm\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.286941 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.286872    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-home\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.286941 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.286900    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-tls-certs\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.287014 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.286945    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58mnh\" (UniqueName: \"kubernetes.io/projected/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-kube-api-access-58mnh\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.388012 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.387982    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-tls-certs\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.388012 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.388015    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58mnh\" (UniqueName: \"kubernetes.io/projected/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-kube-api-access-58mnh\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.388226 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.388047    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-model-cache\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.388226 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.388075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-tmp-dir\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.388226 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.388114    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-dshm\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.388226 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.388133    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-home\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.388567 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.388538    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-home\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.388718 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.388557    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-tmp-dir\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.388718 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.388583    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-model-cache\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.390390 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.390373    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-dshm\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.390640 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.390622    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-tls-certs\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.396059 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.396038    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58mnh\" (UniqueName: \"kubernetes.io/projected/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-kube-api-access-58mnh\") pod \"test-model-738843-kserve-c5cb8b659-hrvwf\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") " pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.512476 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.512384    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:25.648275 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.648250    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"]
Jul 09 14:30:25.650334 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:30:25.650308    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b3ab27c_7e33_4911_b81c_44c4b7b1de14.slice/crio-2f16027c60b77a0d6090cbc0fb05f90cb3b4fe18de0113b884053a062a9b536a WatchSource:0}: Error finding container 2f16027c60b77a0d6090cbc0fb05f90cb3b4fe18de0113b884053a062a9b536a: Status 404 returned error can't find the container with id 2f16027c60b77a0d6090cbc0fb05f90cb3b4fe18de0113b884053a062a9b536a
Jul 09 14:30:25.997505 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.997464    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf" event={"ID":"8b3ab27c-7e33-4911-b81c-44c4b7b1de14","Type":"ContainerStarted","Data":"e4e5adfbdb84446e2fbec798c9a457ae8e14b3d122d4163b26dc87c99d4363e9"}
Jul 09 14:30:25.997505 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:25.997511    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf" event={"ID":"8b3ab27c-7e33-4911-b81c-44c4b7b1de14","Type":"ContainerStarted","Data":"2f16027c60b77a0d6090cbc0fb05f90cb3b4fe18de0113b884053a062a9b536a"}
Jul 09 14:30:26.018331 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:26.018275    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf" podStartSLOduration=1.018258808 podStartE2EDuration="1.018258808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:30:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:30:26.016858143 +0000 UTC m=+2057.799233092" watchObservedRunningTime="2026-07-09 14:30:26.018258808 +0000 UTC m=+2057.800633742"
Jul 09 14:30:35.513049 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:35.512944    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:35.513049 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:35.513016    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:35.525795 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:35.525772    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:30:36.051684 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:30:36.051654    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:31:08.796390 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:08.796359    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 14:31:08.798541 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:08.798520    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 14:31:19.938652 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:19.938608    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"]
Jul 09 14:31:21.978037 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:21.978007    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"]
Jul 09 14:31:21.978459 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:21.978211    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv" podUID="8b105de2-85ef-4694-beee-8d3f40f11390" containerName="maas-api" containerID="cri-o://d16f6a6deb94ba29857bd18b47595bbf3db73e110eee7074a146c85744a37a20" gracePeriod=30
Jul 09 14:31:22.220612 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.220585    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"
Jul 09 14:31:22.238663 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.238569    2579 generic.go:356] "Generic (PLEG): container finished" podID="8b105de2-85ef-4694-beee-8d3f40f11390" containerID="d16f6a6deb94ba29857bd18b47595bbf3db73e110eee7074a146c85744a37a20" exitCode=0
Jul 09 14:31:22.238663 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.238629    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"
Jul 09 14:31:22.238908 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.238656    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv" event={"ID":"8b105de2-85ef-4694-beee-8d3f40f11390","Type":"ContainerDied","Data":"d16f6a6deb94ba29857bd18b47595bbf3db73e110eee7074a146c85744a37a20"}
Jul 09 14:31:22.238908 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.238737    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv" event={"ID":"8b105de2-85ef-4694-beee-8d3f40f11390","Type":"ContainerDied","Data":"cf5164cf7bc4b7d9e65293e041d5bd1fcb8cc8583cbc6b093226fbd18ec97fa1"}
Jul 09 14:31:22.238908 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.238801    2579 scope.go:117] "RemoveContainer" containerID="d16f6a6deb94ba29857bd18b47595bbf3db73e110eee7074a146c85744a37a20"
Jul 09 14:31:22.249547 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.249521    2579 scope.go:117] "RemoveContainer" containerID="d16f6a6deb94ba29857bd18b47595bbf3db73e110eee7074a146c85744a37a20"
Jul 09 14:31:22.249890 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:31:22.249869    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d16f6a6deb94ba29857bd18b47595bbf3db73e110eee7074a146c85744a37a20\": container with ID starting with d16f6a6deb94ba29857bd18b47595bbf3db73e110eee7074a146c85744a37a20 not found: ID does not exist" containerID="d16f6a6deb94ba29857bd18b47595bbf3db73e110eee7074a146c85744a37a20"
Jul 09 14:31:22.249954 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.249903    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d16f6a6deb94ba29857bd18b47595bbf3db73e110eee7074a146c85744a37a20"} err="failed to get container status \"d16f6a6deb94ba29857bd18b47595bbf3db73e110eee7074a146c85744a37a20\": rpc error: code = NotFound desc = could not find container \"d16f6a6deb94ba29857bd18b47595bbf3db73e110eee7074a146c85744a37a20\": container with ID starting with d16f6a6deb94ba29857bd18b47595bbf3db73e110eee7074a146c85744a37a20 not found: ID does not exist"
Jul 09 14:31:22.301564 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.301530    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktc6f\" (UniqueName: \"kubernetes.io/projected/8b105de2-85ef-4694-beee-8d3f40f11390-kube-api-access-ktc6f\") pod \"8b105de2-85ef-4694-beee-8d3f40f11390\" (UID: \"8b105de2-85ef-4694-beee-8d3f40f11390\") "
Jul 09 14:31:22.301773 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.301626    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b105de2-85ef-4694-beee-8d3f40f11390-maas-api-tls\") pod \"8b105de2-85ef-4694-beee-8d3f40f11390\" (UID: \"8b105de2-85ef-4694-beee-8d3f40f11390\") "
Jul 09 14:31:22.301773 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.301667    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b105de2-85ef-4694-beee-8d3f40f11390-openshift-service-ca\") pod \"8b105de2-85ef-4694-beee-8d3f40f11390\" (UID: \"8b105de2-85ef-4694-beee-8d3f40f11390\") "
Jul 09 14:31:22.302078 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.302049    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b105de2-85ef-4694-beee-8d3f40f11390-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8b105de2-85ef-4694-beee-8d3f40f11390" (UID: "8b105de2-85ef-4694-beee-8d3f40f11390"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:31:22.303803 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.303781    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b105de2-85ef-4694-beee-8d3f40f11390-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8b105de2-85ef-4694-beee-8d3f40f11390" (UID: "8b105de2-85ef-4694-beee-8d3f40f11390"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:31:22.303871 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.303840    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b105de2-85ef-4694-beee-8d3f40f11390-kube-api-access-ktc6f" (OuterVolumeSpecName: "kube-api-access-ktc6f") pod "8b105de2-85ef-4694-beee-8d3f40f11390" (UID: "8b105de2-85ef-4694-beee-8d3f40f11390"). InnerVolumeSpecName "kube-api-access-ktc6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:31:22.403301 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.403258    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktc6f\" (UniqueName: \"kubernetes.io/projected/8b105de2-85ef-4694-beee-8d3f40f11390-kube-api-access-ktc6f\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:31:22.403301 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.403301    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b105de2-85ef-4694-beee-8d3f40f11390-maas-api-tls\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:31:22.403519 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.403313    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b105de2-85ef-4694-beee-8d3f40f11390-openshift-service-ca\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:31:22.571133 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.571102    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"]
Jul 09 14:31:22.575476 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.575443    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-738843-89f6cd765-cj5tv"]
Jul 09 14:31:22.825132 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:22.825049    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b105de2-85ef-4694-beee-8d3f40f11390" path="/var/lib/kubelet/pods/8b105de2-85ef-4694-beee-8d3f40f11390/volumes"
Jul 09 14:31:34.961015 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:34.960946    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf" podUID="8b3ab27c-7e33-4911-b81c-44c4b7b1de14" containerName="main" containerID="cri-o://e4e5adfbdb84446e2fbec798c9a457ae8e14b3d122d4163b26dc87c99d4363e9" gracePeriod=45
Jul 09 14:31:35.206992 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.206967    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:31:35.294466 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.294374    2579 generic.go:356] "Generic (PLEG): container finished" podID="8b3ab27c-7e33-4911-b81c-44c4b7b1de14" containerID="e4e5adfbdb84446e2fbec798c9a457ae8e14b3d122d4163b26dc87c99d4363e9" exitCode=0
Jul 09 14:31:35.294466 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.294418    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"
Jul 09 14:31:35.294637 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.294459    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf" event={"ID":"8b3ab27c-7e33-4911-b81c-44c4b7b1de14","Type":"ContainerDied","Data":"e4e5adfbdb84446e2fbec798c9a457ae8e14b3d122d4163b26dc87c99d4363e9"}
Jul 09 14:31:35.294637 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.294505    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf" event={"ID":"8b3ab27c-7e33-4911-b81c-44c4b7b1de14","Type":"ContainerDied","Data":"2f16027c60b77a0d6090cbc0fb05f90cb3b4fe18de0113b884053a062a9b536a"}
Jul 09 14:31:35.294637 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.294523    2579 scope.go:117] "RemoveContainer" containerID="e4e5adfbdb84446e2fbec798c9a457ae8e14b3d122d4163b26dc87c99d4363e9"
Jul 09 14:31:35.302622 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.302603    2579 scope.go:117] "RemoveContainer" containerID="e4e5adfbdb84446e2fbec798c9a457ae8e14b3d122d4163b26dc87c99d4363e9"
Jul 09 14:31:35.302911 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:31:35.302892    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4e5adfbdb84446e2fbec798c9a457ae8e14b3d122d4163b26dc87c99d4363e9\": container with ID starting with e4e5adfbdb84446e2fbec798c9a457ae8e14b3d122d4163b26dc87c99d4363e9 not found: ID does not exist" containerID="e4e5adfbdb84446e2fbec798c9a457ae8e14b3d122d4163b26dc87c99d4363e9"
Jul 09 14:31:35.302970 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.302922    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4e5adfbdb84446e2fbec798c9a457ae8e14b3d122d4163b26dc87c99d4363e9"} err="failed to get container status \"e4e5adfbdb84446e2fbec798c9a457ae8e14b3d122d4163b26dc87c99d4363e9\": rpc error: code = NotFound desc = could not find container \"e4e5adfbdb84446e2fbec798c9a457ae8e14b3d122d4163b26dc87c99d4363e9\": container with ID starting with e4e5adfbdb84446e2fbec798c9a457ae8e14b3d122d4163b26dc87c99d4363e9 not found: ID does not exist"
Jul 09 14:31:35.320763 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.320726    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-tls-certs\") pod \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") "
Jul 09 14:31:35.320876 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.320856    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-dshm\") pod \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") "
Jul 09 14:31:35.320941 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.320881    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-tmp-dir\") pod \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") "
Jul 09 14:31:35.320941 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.320897    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58mnh\" (UniqueName: \"kubernetes.io/projected/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-kube-api-access-58mnh\") pod \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") "
Jul 09 14:31:35.320941 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.320930    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-model-cache\") pod \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") "
Jul 09 14:31:35.321084 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.320953    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-home\") pod \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\" (UID: \"8b3ab27c-7e33-4911-b81c-44c4b7b1de14\") "
Jul 09 14:31:35.321241 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.321185    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8b3ab27c-7e33-4911-b81c-44c4b7b1de14" (UID: "8b3ab27c-7e33-4911-b81c-44c4b7b1de14"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:31:35.321351 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.321241    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-model-cache" (OuterVolumeSpecName: "model-cache") pod "8b3ab27c-7e33-4911-b81c-44c4b7b1de14" (UID: "8b3ab27c-7e33-4911-b81c-44c4b7b1de14"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:31:35.321351 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.321250    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-home" (OuterVolumeSpecName: "home") pod "8b3ab27c-7e33-4911-b81c-44c4b7b1de14" (UID: "8b3ab27c-7e33-4911-b81c-44c4b7b1de14"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:31:35.323213 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.323174    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-kube-api-access-58mnh" (OuterVolumeSpecName: "kube-api-access-58mnh") pod "8b3ab27c-7e33-4911-b81c-44c4b7b1de14" (UID: "8b3ab27c-7e33-4911-b81c-44c4b7b1de14"). InnerVolumeSpecName "kube-api-access-58mnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:31:35.323213 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.323196    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-dshm" (OuterVolumeSpecName: "dshm") pod "8b3ab27c-7e33-4911-b81c-44c4b7b1de14" (UID: "8b3ab27c-7e33-4911-b81c-44c4b7b1de14"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:31:35.323213 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.323191    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8b3ab27c-7e33-4911-b81c-44c4b7b1de14" (UID: "8b3ab27c-7e33-4911-b81c-44c4b7b1de14"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:31:35.422463 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.422428    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-dshm\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:31:35.422463 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.422457    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-tmp-dir\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:31:35.422463 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.422468    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58mnh\" (UniqueName: \"kubernetes.io/projected/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-kube-api-access-58mnh\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:31:35.422717 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.422477    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-model-cache\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:31:35.422717 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.422485    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-home\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:31:35.422717 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.422494    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b3ab27c-7e33-4911-b81c-44c4b7b1de14-tls-certs\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:31:35.616560 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.616522    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"]
Jul 09 14:31:35.620470 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:35.620441    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-738843/test-model-738843-kserve-c5cb8b659-hrvwf"]
Jul 09 14:31:36.824212 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:31:36.824173    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b3ab27c-7e33-4911-b81c-44c4b7b1de14" path="/var/lib/kubelet/pods/8b3ab27c-7e33-4911-b81c-44c4b7b1de14/volumes"
Jul 09 14:32:05.371453 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:05.371422    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"]
Jul 09 14:32:05.372008 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:05.371669    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj" podUID="6a58756f-fc4c-41cd-81f0-6276644415af" containerName="maas-api" containerID="cri-o://29491b8ff3fc46a4891776abd308a7be17b875516728915d9317d6f3b418546a" gracePeriod=30
Jul 09 14:32:05.621797 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:05.621712    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:32:05.668151 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:05.668121    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a58756f-fc4c-41cd-81f0-6276644415af-maas-api-tls\") pod \"6a58756f-fc4c-41cd-81f0-6276644415af\" (UID: \"6a58756f-fc4c-41cd-81f0-6276644415af\") "
Jul 09 14:32:05.668315 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:05.668179    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx65w\" (UniqueName: \"kubernetes.io/projected/6a58756f-fc4c-41cd-81f0-6276644415af-kube-api-access-hx65w\") pod \"6a58756f-fc4c-41cd-81f0-6276644415af\" (UID: \"6a58756f-fc4c-41cd-81f0-6276644415af\") "
Jul 09 14:32:05.668371 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:05.668308    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a58756f-fc4c-41cd-81f0-6276644415af-openshift-service-ca\") pod \"6a58756f-fc4c-41cd-81f0-6276644415af\" (UID: \"6a58756f-fc4c-41cd-81f0-6276644415af\") "
Jul 09 14:32:05.668685 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:05.668658    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a58756f-fc4c-41cd-81f0-6276644415af-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6a58756f-fc4c-41cd-81f0-6276644415af" (UID: "6a58756f-fc4c-41cd-81f0-6276644415af"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:32:05.670299 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:05.670276    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a58756f-fc4c-41cd-81f0-6276644415af-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6a58756f-fc4c-41cd-81f0-6276644415af" (UID: "6a58756f-fc4c-41cd-81f0-6276644415af"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:32:05.670382 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:05.670356    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a58756f-fc4c-41cd-81f0-6276644415af-kube-api-access-hx65w" (OuterVolumeSpecName: "kube-api-access-hx65w") pod "6a58756f-fc4c-41cd-81f0-6276644415af" (UID: "6a58756f-fc4c-41cd-81f0-6276644415af"). InnerVolumeSpecName "kube-api-access-hx65w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:32:05.769509 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:05.769475    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a58756f-fc4c-41cd-81f0-6276644415af-maas-api-tls\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:32:05.769509 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:05.769502    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx65w\" (UniqueName: \"kubernetes.io/projected/6a58756f-fc4c-41cd-81f0-6276644415af-kube-api-access-hx65w\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:32:05.769509 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:05.769513    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a58756f-fc4c-41cd-81f0-6276644415af-openshift-service-ca\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:32:06.433270 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:06.433225    2579 generic.go:356] "Generic (PLEG): container finished" podID="6a58756f-fc4c-41cd-81f0-6276644415af" containerID="29491b8ff3fc46a4891776abd308a7be17b875516728915d9317d6f3b418546a" exitCode=0
Jul 09 14:32:06.433727 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:06.433328    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj" event={"ID":"6a58756f-fc4c-41cd-81f0-6276644415af","Type":"ContainerDied","Data":"29491b8ff3fc46a4891776abd308a7be17b875516728915d9317d6f3b418546a"}
Jul 09 14:32:06.433727 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:06.433419    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj" event={"ID":"6a58756f-fc4c-41cd-81f0-6276644415af","Type":"ContainerDied","Data":"5af386a5a4beb0ced5091a6e7fb70037a67a89beed8256a1b1861e00bb6273a5"}
Jul 09 14:32:06.433727 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:06.433453    2579 scope.go:117] "RemoveContainer" containerID="29491b8ff3fc46a4891776abd308a7be17b875516728915d9317d6f3b418546a"
Jul 09 14:32:06.433727 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:06.433652    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"
Jul 09 14:32:06.443450 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:06.443426    2579 scope.go:117] "RemoveContainer" containerID="29491b8ff3fc46a4891776abd308a7be17b875516728915d9317d6f3b418546a"
Jul 09 14:32:06.443711 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:32:06.443692    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29491b8ff3fc46a4891776abd308a7be17b875516728915d9317d6f3b418546a\": container with ID starting with 29491b8ff3fc46a4891776abd308a7be17b875516728915d9317d6f3b418546a not found: ID does not exist" containerID="29491b8ff3fc46a4891776abd308a7be17b875516728915d9317d6f3b418546a"
Jul 09 14:32:06.443800 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:06.443720    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29491b8ff3fc46a4891776abd308a7be17b875516728915d9317d6f3b418546a"} err="failed to get container status \"29491b8ff3fc46a4891776abd308a7be17b875516728915d9317d6f3b418546a\": rpc error: code = NotFound desc = could not find container \"29491b8ff3fc46a4891776abd308a7be17b875516728915d9317d6f3b418546a\": container with ID starting with 29491b8ff3fc46a4891776abd308a7be17b875516728915d9317d6f3b418546a not found: ID does not exist"
Jul 09 14:32:06.456268 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:06.456241    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"]
Jul 09 14:32:06.463026 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:06.463005    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f1a2e7-5d849cc947-m96dj"]
Jul 09 14:32:06.824902 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:06.824815    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a58756f-fc4c-41cd-81f0-6276644415af" path="/var/lib/kubelet/pods/6a58756f-fc4c-41cd-81f0-6276644415af/volumes"
Jul 09 14:32:15.547243 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.547211    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"]
Jul 09 14:32:15.547708 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.547527    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b105de2-85ef-4694-beee-8d3f40f11390" containerName="maas-api"
Jul 09 14:32:15.547708 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.547541    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b105de2-85ef-4694-beee-8d3f40f11390" containerName="maas-api"
Jul 09 14:32:15.547708 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.547561    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a58756f-fc4c-41cd-81f0-6276644415af" containerName="maas-api"
Jul 09 14:32:15.547708 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.547568    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a58756f-fc4c-41cd-81f0-6276644415af" containerName="maas-api"
Jul 09 14:32:15.547708 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.547582    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b3ab27c-7e33-4911-b81c-44c4b7b1de14" containerName="main"
Jul 09 14:32:15.547708 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.547587    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b3ab27c-7e33-4911-b81c-44c4b7b1de14" containerName="main"
Jul 09 14:32:15.547708 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.547643    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b3ab27c-7e33-4911-b81c-44c4b7b1de14" containerName="main"
Jul 09 14:32:15.547708 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.547651    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b105de2-85ef-4694-beee-8d3f40f11390" containerName="maas-api"
Jul 09 14:32:15.547708 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.547657    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a58756f-fc4c-41cd-81f0-6276644415af" containerName="maas-api"
Jul 09 14:32:15.550577 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.550563    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"
Jul 09 14:32:15.555304 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.555285    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-38f05c-serving-cert\""
Jul 09 14:32:15.564473 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.564449    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"]
Jul 09 14:32:15.642816 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.642774    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-openshift-service-ca\") pod \"maas-api-e2e-api-a-38f05c-9879846d8-g7vsz\" (UID: \"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"
Jul 09 14:32:15.642993 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.642892    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwgh7\" (UniqueName: \"kubernetes.io/projected/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-kube-api-access-bwgh7\") pod \"maas-api-e2e-api-a-38f05c-9879846d8-g7vsz\" (UID: \"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"
Jul 09 14:32:15.642993 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.642927    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-maas-api-tls\") pod \"maas-api-e2e-api-a-38f05c-9879846d8-g7vsz\" (UID: \"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"
Jul 09 14:32:15.744188 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.744146    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwgh7\" (UniqueName: \"kubernetes.io/projected/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-kube-api-access-bwgh7\") pod \"maas-api-e2e-api-a-38f05c-9879846d8-g7vsz\" (UID: \"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"
Jul 09 14:32:15.744188 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.744187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-maas-api-tls\") pod \"maas-api-e2e-api-a-38f05c-9879846d8-g7vsz\" (UID: \"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"
Jul 09 14:32:15.744390 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.744221    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-openshift-service-ca\") pod \"maas-api-e2e-api-a-38f05c-9879846d8-g7vsz\" (UID: \"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"
Jul 09 14:32:15.744390 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:32:15.744302    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-serving-cert: secret "maas-api-e2e-api-a-38f05c-serving-cert" not found
Jul 09 14:32:15.744390 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:32:15.744370    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-maas-api-tls podName:e3d8aaa2-f6af-40b9-a4d9-b19f638607e2 nodeName:}" failed. No retries permitted until 2026-07-09 14:32:16.244351099 +0000 UTC m=+2168.026726011 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-maas-api-tls") pod "maas-api-e2e-api-a-38f05c-9879846d8-g7vsz" (UID: "e3d8aaa2-f6af-40b9-a4d9-b19f638607e2") : secret "maas-api-e2e-api-a-38f05c-serving-cert" not found
Jul 09 14:32:15.744819 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.744803    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-openshift-service-ca\") pod \"maas-api-e2e-api-a-38f05c-9879846d8-g7vsz\" (UID: \"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"
Jul 09 14:32:15.812077 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:15.812005    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwgh7\" (UniqueName: \"kubernetes.io/projected/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-kube-api-access-bwgh7\") pod \"maas-api-e2e-api-a-38f05c-9879846d8-g7vsz\" (UID: \"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"
Jul 09 14:32:16.248391 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:16.248356    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-maas-api-tls\") pod \"maas-api-e2e-api-a-38f05c-9879846d8-g7vsz\" (UID: \"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"
Jul 09 14:32:16.250922 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:16.250897    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-maas-api-tls\") pod \"maas-api-e2e-api-a-38f05c-9879846d8-g7vsz\" (UID: \"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"
Jul 09 14:32:16.459982 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:16.459953    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"
Jul 09 14:32:16.601278 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:16.601179    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"]
Jul 09 14:32:16.604211 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:32:16.604183    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3d8aaa2_f6af_40b9_a4d9_b19f638607e2.slice/crio-e73a3f4ff896b82fabf378f1effeff030f3e2c04de5a9a1b4ef674489775025b WatchSource:0}: Error finding container e73a3f4ff896b82fabf378f1effeff030f3e2c04de5a9a1b4ef674489775025b: Status 404 returned error can't find the container with id e73a3f4ff896b82fabf378f1effeff030f3e2c04de5a9a1b4ef674489775025b
Jul 09 14:32:17.481109 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:17.481073    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz" event={"ID":"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2","Type":"ContainerStarted","Data":"e73a3f4ff896b82fabf378f1effeff030f3e2c04de5a9a1b4ef674489775025b"}
Jul 09 14:32:18.486840 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:18.486805    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz" event={"ID":"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2","Type":"ContainerStarted","Data":"0d58130ed9ea23025f5c80c67e855429f3e58ef6c1881e50fe41a73f48dc60b8"}
Jul 09 14:32:18.487228 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:18.486940    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"
Jul 09 14:32:18.504804 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:18.504731    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz" podStartSLOduration=2.422205692 podStartE2EDuration="3.50471075s" totalImagesPullingTime="1.082505058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:32:18.503982782 +0000 UTC m=+2170.286357717" watchObservedRunningTime="2026-07-09 14:32:18.50471075 +0000 UTC m=+2170.287085689"
Jul 09 14:32:23.596020 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:23.595978    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"]
Jul 09 14:32:23.600041 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:23.600014    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:23.604664 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:23.604628    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-7c2ea1-serving-cert\""
Jul 09 14:32:23.614559 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:23.614530    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"]
Jul 09 14:32:23.715529 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:23.715494    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frzwp\" (UniqueName: \"kubernetes.io/projected/10590d45-8a4b-4b87-b963-ed45a7a22e75-kube-api-access-frzwp\") pod \"maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p\" (UID: \"10590d45-8a4b-4b87-b963-ed45a7a22e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:23.715529 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:23.715532    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10590d45-8a4b-4b87-b963-ed45a7a22e75-maas-api-tls\") pod \"maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p\" (UID: \"10590d45-8a4b-4b87-b963-ed45a7a22e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:23.715761 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:23.715623    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10590d45-8a4b-4b87-b963-ed45a7a22e75-openshift-service-ca\") pod \"maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p\" (UID: \"10590d45-8a4b-4b87-b963-ed45a7a22e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:23.816358 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:23.816324    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10590d45-8a4b-4b87-b963-ed45a7a22e75-openshift-service-ca\") pod \"maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p\" (UID: \"10590d45-8a4b-4b87-b963-ed45a7a22e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:23.816535 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:23.816375    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frzwp\" (UniqueName: \"kubernetes.io/projected/10590d45-8a4b-4b87-b963-ed45a7a22e75-kube-api-access-frzwp\") pod \"maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p\" (UID: \"10590d45-8a4b-4b87-b963-ed45a7a22e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:23.816535 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:23.816496    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10590d45-8a4b-4b87-b963-ed45a7a22e75-maas-api-tls\") pod \"maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p\" (UID: \"10590d45-8a4b-4b87-b963-ed45a7a22e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:23.816617 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:32:23.816608    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-serving-cert: secret "maas-api-e2e-api-b-7c2ea1-serving-cert" not found
Jul 09 14:32:23.816684 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:32:23.816674    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10590d45-8a4b-4b87-b963-ed45a7a22e75-maas-api-tls podName:10590d45-8a4b-4b87-b963-ed45a7a22e75 nodeName:}" failed. No retries permitted until 2026-07-09 14:32:24.316656489 +0000 UTC m=+2176.099031402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/10590d45-8a4b-4b87-b963-ed45a7a22e75-maas-api-tls") pod "maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p" (UID: "10590d45-8a4b-4b87-b963-ed45a7a22e75") : secret "maas-api-e2e-api-b-7c2ea1-serving-cert" not found
Jul 09 14:32:23.817018 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:23.817001    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10590d45-8a4b-4b87-b963-ed45a7a22e75-openshift-service-ca\") pod \"maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p\" (UID: \"10590d45-8a4b-4b87-b963-ed45a7a22e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:23.827470 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:23.827446    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frzwp\" (UniqueName: \"kubernetes.io/projected/10590d45-8a4b-4b87-b963-ed45a7a22e75-kube-api-access-frzwp\") pod \"maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p\" (UID: \"10590d45-8a4b-4b87-b963-ed45a7a22e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:24.322032 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:24.321996    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10590d45-8a4b-4b87-b963-ed45a7a22e75-maas-api-tls\") pod \"maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p\" (UID: \"10590d45-8a4b-4b87-b963-ed45a7a22e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:24.322221 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:32:24.322146    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-serving-cert: secret "maas-api-e2e-api-b-7c2ea1-serving-cert" not found
Jul 09 14:32:24.322221 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:32:24.322210    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10590d45-8a4b-4b87-b963-ed45a7a22e75-maas-api-tls podName:10590d45-8a4b-4b87-b963-ed45a7a22e75 nodeName:}" failed. No retries permitted until 2026-07-09 14:32:25.322192552 +0000 UTC m=+2177.104567467 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/10590d45-8a4b-4b87-b963-ed45a7a22e75-maas-api-tls") pod "maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p" (UID: "10590d45-8a4b-4b87-b963-ed45a7a22e75") : secret "maas-api-e2e-api-b-7c2ea1-serving-cert" not found
Jul 09 14:32:24.498299 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:24.498270    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"
Jul 09 14:32:25.331174 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:25.331138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10590d45-8a4b-4b87-b963-ed45a7a22e75-maas-api-tls\") pod \"maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p\" (UID: \"10590d45-8a4b-4b87-b963-ed45a7a22e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:25.333760 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:25.333728    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10590d45-8a4b-4b87-b963-ed45a7a22e75-maas-api-tls\") pod \"maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p\" (UID: \"10590d45-8a4b-4b87-b963-ed45a7a22e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:25.417478 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:25.417438    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:25.550978 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:25.550949    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"]
Jul 09 14:32:25.553367 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:32:25.553339    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10590d45_8a4b_4b87_b963_ed45a7a22e75.slice/crio-cfd20ead63ad7d04d6daa15dd47ba4237b8e3b1947412af6249708e17fe96b68 WatchSource:0}: Error finding container cfd20ead63ad7d04d6daa15dd47ba4237b8e3b1947412af6249708e17fe96b68: Status 404 returned error can't find the container with id cfd20ead63ad7d04d6daa15dd47ba4237b8e3b1947412af6249708e17fe96b68
Jul 09 14:32:26.523951 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:26.523909    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p" event={"ID":"10590d45-8a4b-4b87-b963-ed45a7a22e75","Type":"ContainerStarted","Data":"8b9bf96e1ad587ea99546205b042fbc90e4746d39f11bf0b72eb557f2027032c"}
Jul 09 14:32:26.524377 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:26.523968    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p" event={"ID":"10590d45-8a4b-4b87-b963-ed45a7a22e75","Type":"ContainerStarted","Data":"cfd20ead63ad7d04d6daa15dd47ba4237b8e3b1947412af6249708e17fe96b68"}
Jul 09 14:32:26.524377 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:26.524074    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:26.548596 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:26.548548    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p" podStartSLOduration=3.244551611 podStartE2EDuration="3.548530706s" totalImagesPullingTime="303.979095ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:32:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:32:26.545952251 +0000 UTC m=+2178.328327185" watchObservedRunningTime="2026-07-09 14:32:26.548530706 +0000 UTC m=+2178.330905640"
Jul 09 14:32:32.533567 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:32.533537    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:36.895869 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:36.895832    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"]
Jul 09 14:32:36.896287 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:36.896097    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz" podUID="e3d8aaa2-f6af-40b9-a4d9-b19f638607e2" containerName="maas-api" containerID="cri-o://0d58130ed9ea23025f5c80c67e855429f3e58ef6c1881e50fe41a73f48dc60b8" gracePeriod=30
Jul 09 14:32:37.144293 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.144264    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"
Jul 09 14:32:37.240730 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.240635    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-maas-api-tls\") pod \"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2\" (UID: \"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2\") "
Jul 09 14:32:37.240930 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.240728    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-openshift-service-ca\") pod \"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2\" (UID: \"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2\") "
Jul 09 14:32:37.240930 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.240792    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwgh7\" (UniqueName: \"kubernetes.io/projected/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-kube-api-access-bwgh7\") pod \"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2\" (UID: \"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2\") "
Jul 09 14:32:37.241147 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.241124    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e3d8aaa2-f6af-40b9-a4d9-b19f638607e2" (UID: "e3d8aaa2-f6af-40b9-a4d9-b19f638607e2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:32:37.242960 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.242938    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-kube-api-access-bwgh7" (OuterVolumeSpecName: "kube-api-access-bwgh7") pod "e3d8aaa2-f6af-40b9-a4d9-b19f638607e2" (UID: "e3d8aaa2-f6af-40b9-a4d9-b19f638607e2"). InnerVolumeSpecName "kube-api-access-bwgh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:32:37.243052 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.242972    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e3d8aaa2-f6af-40b9-a4d9-b19f638607e2" (UID: "e3d8aaa2-f6af-40b9-a4d9-b19f638607e2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:32:37.341484 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.341442    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-openshift-service-ca\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:32:37.341484 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.341479    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwgh7\" (UniqueName: \"kubernetes.io/projected/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-kube-api-access-bwgh7\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:32:37.341484 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.341488    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2-maas-api-tls\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:32:37.574281 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.574195    2579 generic.go:356] "Generic (PLEG): container finished" podID="e3d8aaa2-f6af-40b9-a4d9-b19f638607e2" containerID="0d58130ed9ea23025f5c80c67e855429f3e58ef6c1881e50fe41a73f48dc60b8" exitCode=0
Jul 09 14:32:37.574281 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.574241    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"
Jul 09 14:32:37.574499 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.574278    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz" event={"ID":"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2","Type":"ContainerDied","Data":"0d58130ed9ea23025f5c80c67e855429f3e58ef6c1881e50fe41a73f48dc60b8"}
Jul 09 14:32:37.574499 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.574330    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz" event={"ID":"e3d8aaa2-f6af-40b9-a4d9-b19f638607e2","Type":"ContainerDied","Data":"e73a3f4ff896b82fabf378f1effeff030f3e2c04de5a9a1b4ef674489775025b"}
Jul 09 14:32:37.574499 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.574353    2579 scope.go:117] "RemoveContainer" containerID="0d58130ed9ea23025f5c80c67e855429f3e58ef6c1881e50fe41a73f48dc60b8"
Jul 09 14:32:37.583990 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.583975    2579 scope.go:117] "RemoveContainer" containerID="0d58130ed9ea23025f5c80c67e855429f3e58ef6c1881e50fe41a73f48dc60b8"
Jul 09 14:32:37.584289 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:32:37.584267    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d58130ed9ea23025f5c80c67e855429f3e58ef6c1881e50fe41a73f48dc60b8\": container with ID starting with 0d58130ed9ea23025f5c80c67e855429f3e58ef6c1881e50fe41a73f48dc60b8 not found: ID does not exist" containerID="0d58130ed9ea23025f5c80c67e855429f3e58ef6c1881e50fe41a73f48dc60b8"
Jul 09 14:32:37.584344 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.584299    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d58130ed9ea23025f5c80c67e855429f3e58ef6c1881e50fe41a73f48dc60b8"} err="failed to get container status \"0d58130ed9ea23025f5c80c67e855429f3e58ef6c1881e50fe41a73f48dc60b8\": rpc error: code = NotFound desc = could not find container \"0d58130ed9ea23025f5c80c67e855429f3e58ef6c1881e50fe41a73f48dc60b8\": container with ID starting with 0d58130ed9ea23025f5c80c67e855429f3e58ef6c1881e50fe41a73f48dc60b8 not found: ID does not exist"
Jul 09 14:32:37.595470 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.595445    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"]
Jul 09 14:32:37.599657 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:37.599632    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-38f05c-9879846d8-g7vsz"]
Jul 09 14:32:38.825926 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:38.825896    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3d8aaa2-f6af-40b9-a4d9-b19f638607e2" path="/var/lib/kubelet/pods/e3d8aaa2-f6af-40b9-a4d9-b19f638607e2/volumes"
Jul 09 14:32:44.791732 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:44.791692    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"]
Jul 09 14:32:44.792153 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:44.792003    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p" podUID="10590d45-8a4b-4b87-b963-ed45a7a22e75" containerName="maas-api" containerID="cri-o://8b9bf96e1ad587ea99546205b042fbc90e4746d39f11bf0b72eb557f2027032c" gracePeriod=30
Jul 09 14:32:45.034605 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.034583    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:45.111188 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.111154    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frzwp\" (UniqueName: \"kubernetes.io/projected/10590d45-8a4b-4b87-b963-ed45a7a22e75-kube-api-access-frzwp\") pod \"10590d45-8a4b-4b87-b963-ed45a7a22e75\" (UID: \"10590d45-8a4b-4b87-b963-ed45a7a22e75\") "
Jul 09 14:32:45.111385 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.111212    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10590d45-8a4b-4b87-b963-ed45a7a22e75-maas-api-tls\") pod \"10590d45-8a4b-4b87-b963-ed45a7a22e75\" (UID: \"10590d45-8a4b-4b87-b963-ed45a7a22e75\") "
Jul 09 14:32:45.111385 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.111250    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10590d45-8a4b-4b87-b963-ed45a7a22e75-openshift-service-ca\") pod \"10590d45-8a4b-4b87-b963-ed45a7a22e75\" (UID: \"10590d45-8a4b-4b87-b963-ed45a7a22e75\") "
Jul 09 14:32:45.111696 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.111668    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10590d45-8a4b-4b87-b963-ed45a7a22e75-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "10590d45-8a4b-4b87-b963-ed45a7a22e75" (UID: "10590d45-8a4b-4b87-b963-ed45a7a22e75"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:32:45.113368 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.113334    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10590d45-8a4b-4b87-b963-ed45a7a22e75-kube-api-access-frzwp" (OuterVolumeSpecName: "kube-api-access-frzwp") pod "10590d45-8a4b-4b87-b963-ed45a7a22e75" (UID: "10590d45-8a4b-4b87-b963-ed45a7a22e75"). InnerVolumeSpecName "kube-api-access-frzwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:32:45.113465 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.113413    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10590d45-8a4b-4b87-b963-ed45a7a22e75-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "10590d45-8a4b-4b87-b963-ed45a7a22e75" (UID: "10590d45-8a4b-4b87-b963-ed45a7a22e75"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:32:45.212464 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.212425    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10590d45-8a4b-4b87-b963-ed45a7a22e75-maas-api-tls\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:32:45.212464 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.212457    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10590d45-8a4b-4b87-b963-ed45a7a22e75-openshift-service-ca\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:32:45.212464 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.212469    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frzwp\" (UniqueName: \"kubernetes.io/projected/10590d45-8a4b-4b87-b963-ed45a7a22e75-kube-api-access-frzwp\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:32:45.611930 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.611893    2579 generic.go:356] "Generic (PLEG): container finished" podID="10590d45-8a4b-4b87-b963-ed45a7a22e75" containerID="8b9bf96e1ad587ea99546205b042fbc90e4746d39f11bf0b72eb557f2027032c" exitCode=0
Jul 09 14:32:45.611930 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.611935    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p" event={"ID":"10590d45-8a4b-4b87-b963-ed45a7a22e75","Type":"ContainerDied","Data":"8b9bf96e1ad587ea99546205b042fbc90e4746d39f11bf0b72eb557f2027032c"}
Jul 09 14:32:45.612209 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.611943    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"
Jul 09 14:32:45.612209 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.611987    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p" event={"ID":"10590d45-8a4b-4b87-b963-ed45a7a22e75","Type":"ContainerDied","Data":"cfd20ead63ad7d04d6daa15dd47ba4237b8e3b1947412af6249708e17fe96b68"}
Jul 09 14:32:45.612209 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.612020    2579 scope.go:117] "RemoveContainer" containerID="8b9bf96e1ad587ea99546205b042fbc90e4746d39f11bf0b72eb557f2027032c"
Jul 09 14:32:45.621252 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.621230    2579 scope.go:117] "RemoveContainer" containerID="8b9bf96e1ad587ea99546205b042fbc90e4746d39f11bf0b72eb557f2027032c"
Jul 09 14:32:45.621525 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:32:45.621504    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b9bf96e1ad587ea99546205b042fbc90e4746d39f11bf0b72eb557f2027032c\": container with ID starting with 8b9bf96e1ad587ea99546205b042fbc90e4746d39f11bf0b72eb557f2027032c not found: ID does not exist" containerID="8b9bf96e1ad587ea99546205b042fbc90e4746d39f11bf0b72eb557f2027032c"
Jul 09 14:32:45.621580 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.621534    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9bf96e1ad587ea99546205b042fbc90e4746d39f11bf0b72eb557f2027032c"} err="failed to get container status \"8b9bf96e1ad587ea99546205b042fbc90e4746d39f11bf0b72eb557f2027032c\": rpc error: code = NotFound desc = could not find container \"8b9bf96e1ad587ea99546205b042fbc90e4746d39f11bf0b72eb557f2027032c\": container with ID starting with 8b9bf96e1ad587ea99546205b042fbc90e4746d39f11bf0b72eb557f2027032c not found: ID does not exist"
Jul 09 14:32:45.633895 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.633872    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"]
Jul 09 14:32:45.636412 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:45.636391    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7c2ea1-54474d4cb9-kw88p"]
Jul 09 14:32:46.824511 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:46.824475    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10590d45-8a4b-4b87-b963-ed45a7a22e75" path="/var/lib/kubelet/pods/10590d45-8a4b-4b87-b963-ed45a7a22e75/volumes"
Jul 09 14:32:54.792688 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:54.792647    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"]
Jul 09 14:32:54.793294 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:54.793136    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10590d45-8a4b-4b87-b963-ed45a7a22e75" containerName="maas-api"
Jul 09 14:32:54.793294 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:54.793155    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="10590d45-8a4b-4b87-b963-ed45a7a22e75" containerName="maas-api"
Jul 09 14:32:54.793294 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:54.793181    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3d8aaa2-f6af-40b9-a4d9-b19f638607e2" containerName="maas-api"
Jul 09 14:32:54.793294 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:54.793191    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3d8aaa2-f6af-40b9-a4d9-b19f638607e2" containerName="maas-api"
Jul 09 14:32:54.793294 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:54.793263    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="10590d45-8a4b-4b87-b963-ed45a7a22e75" containerName="maas-api"
Jul 09 14:32:54.793294 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:54.793275    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3d8aaa2-f6af-40b9-a4d9-b19f638607e2" containerName="maas-api"
Jul 09 14:32:54.796136 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:54.796115    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"
Jul 09 14:32:54.798775 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:54.798737    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-6b90ba-serving-cert\""
Jul 09 14:32:54.811614 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:54.811575    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"]
Jul 09 14:32:54.899532 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:54.899485    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78cbecfa-d7a2-4093-98fc-71fb4c30046f-maas-api-tls\") pod \"maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t\" (UID: \"78cbecfa-d7a2-4093-98fc-71fb4c30046f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"
Jul 09 14:32:54.899719 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:54.899552    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78cbecfa-d7a2-4093-98fc-71fb4c30046f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t\" (UID: \"78cbecfa-d7a2-4093-98fc-71fb4c30046f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"
Jul 09 14:32:54.899719 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:54.899657    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlbkm\" (UniqueName: \"kubernetes.io/projected/78cbecfa-d7a2-4093-98fc-71fb4c30046f-kube-api-access-mlbkm\") pod \"maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t\" (UID: \"78cbecfa-d7a2-4093-98fc-71fb4c30046f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"
Jul 09 14:32:55.000076 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:55.000036    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78cbecfa-d7a2-4093-98fc-71fb4c30046f-maas-api-tls\") pod \"maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t\" (UID: \"78cbecfa-d7a2-4093-98fc-71fb4c30046f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"
Jul 09 14:32:55.000254 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:55.000088    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78cbecfa-d7a2-4093-98fc-71fb4c30046f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t\" (UID: \"78cbecfa-d7a2-4093-98fc-71fb4c30046f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"
Jul 09 14:32:55.000254 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:55.000150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlbkm\" (UniqueName: \"kubernetes.io/projected/78cbecfa-d7a2-4093-98fc-71fb4c30046f-kube-api-access-mlbkm\") pod \"maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t\" (UID: \"78cbecfa-d7a2-4093-98fc-71fb4c30046f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"
Jul 09 14:32:55.000254 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:32:55.000205    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-serving-cert: secret "maas-api-e2e-shared-a-6b90ba-serving-cert" not found
Jul 09 14:32:55.000382 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:32:55.000300    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78cbecfa-d7a2-4093-98fc-71fb4c30046f-maas-api-tls podName:78cbecfa-d7a2-4093-98fc-71fb4c30046f nodeName:}" failed. No retries permitted until 2026-07-09 14:32:55.500279717 +0000 UTC m=+2207.282654659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/78cbecfa-d7a2-4093-98fc-71fb4c30046f-maas-api-tls") pod "maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t" (UID: "78cbecfa-d7a2-4093-98fc-71fb4c30046f") : secret "maas-api-e2e-shared-a-6b90ba-serving-cert" not found
Jul 09 14:32:55.000843 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:55.000825    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78cbecfa-d7a2-4093-98fc-71fb4c30046f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t\" (UID: \"78cbecfa-d7a2-4093-98fc-71fb4c30046f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"
Jul 09 14:32:55.013282 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:55.013250    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlbkm\" (UniqueName: \"kubernetes.io/projected/78cbecfa-d7a2-4093-98fc-71fb4c30046f-kube-api-access-mlbkm\") pod \"maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t\" (UID: \"78cbecfa-d7a2-4093-98fc-71fb4c30046f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"
Jul 09 14:32:55.506878 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:55.506847    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78cbecfa-d7a2-4093-98fc-71fb4c30046f-maas-api-tls\") pod \"maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t\" (UID: \"78cbecfa-d7a2-4093-98fc-71fb4c30046f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"
Jul 09 14:32:55.509369 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:55.509340    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78cbecfa-d7a2-4093-98fc-71fb4c30046f-maas-api-tls\") pod \"maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t\" (UID: \"78cbecfa-d7a2-4093-98fc-71fb4c30046f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"
Jul 09 14:32:55.706335 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:55.706297    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"
Jul 09 14:32:55.843424 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:55.843391    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"]
Jul 09 14:32:55.845881 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:32:55.845854    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78cbecfa_d7a2_4093_98fc_71fb4c30046f.slice/crio-0bf0ef9d7f1db96e0ea312e142a75b9832e5cfca352e04dd32e642acc3036d48 WatchSource:0}: Error finding container 0bf0ef9d7f1db96e0ea312e142a75b9832e5cfca352e04dd32e642acc3036d48: Status 404 returned error can't find the container with id 0bf0ef9d7f1db96e0ea312e142a75b9832e5cfca352e04dd32e642acc3036d48
Jul 09 14:32:56.660966 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:56.660933    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t" event={"ID":"78cbecfa-d7a2-4093-98fc-71fb4c30046f","Type":"ContainerStarted","Data":"ff95deac1be2987528e2bb23c48b96a561d22ff8f51ee215a24d82d9fd18c9e6"}
Jul 09 14:32:56.661169 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:56.660996    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t" event={"ID":"78cbecfa-d7a2-4093-98fc-71fb4c30046f","Type":"ContainerStarted","Data":"0bf0ef9d7f1db96e0ea312e142a75b9832e5cfca352e04dd32e642acc3036d48"}
Jul 09 14:32:56.661169 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:56.661024    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"
Jul 09 14:32:56.684415 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:32:56.684367    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t" podStartSLOduration=2.3545434849999998 podStartE2EDuration="2.684349123s" totalImagesPullingTime="329.805638ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:32:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:32:56.682780286 +0000 UTC m=+2208.465155218" watchObservedRunningTime="2026-07-09 14:32:56.684349123 +0000 UTC m=+2208.466724083"
Jul 09 14:33:02.670825 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:02.670796    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"
Jul 09 14:33:02.912386 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:02.912353    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"]
Jul 09 14:33:02.915873 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:02.915840    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"
Jul 09 14:33:02.918639 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:02.918619    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-11d0ca-serving-cert\""
Jul 09 14:33:02.942689 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:02.942647    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"]
Jul 09 14:33:02.969869 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:02.969835    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hgfx\" (UniqueName: \"kubernetes.io/projected/7f0b32df-989a-49c5-bf26-420af69315f3-kube-api-access-2hgfx\") pod \"maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f\" (UID: \"7f0b32df-989a-49c5-bf26-420af69315f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"
Jul 09 14:33:02.970036 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:02.969898    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f0b32df-989a-49c5-bf26-420af69315f3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f\" (UID: \"7f0b32df-989a-49c5-bf26-420af69315f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"
Jul 09 14:33:02.970036 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:02.969980    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f0b32df-989a-49c5-bf26-420af69315f3-maas-api-tls\") pod \"maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f\" (UID: \"7f0b32df-989a-49c5-bf26-420af69315f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"
Jul 09 14:33:03.070839 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:03.070796    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hgfx\" (UniqueName: \"kubernetes.io/projected/7f0b32df-989a-49c5-bf26-420af69315f3-kube-api-access-2hgfx\") pod \"maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f\" (UID: \"7f0b32df-989a-49c5-bf26-420af69315f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"
Jul 09 14:33:03.071180 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:03.070879    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f0b32df-989a-49c5-bf26-420af69315f3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f\" (UID: \"7f0b32df-989a-49c5-bf26-420af69315f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"
Jul 09 14:33:03.071180 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:03.070915    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f0b32df-989a-49c5-bf26-420af69315f3-maas-api-tls\") pod \"maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f\" (UID: \"7f0b32df-989a-49c5-bf26-420af69315f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"
Jul 09 14:33:03.071180 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:33:03.071040    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-serving-cert: secret "maas-api-e2e-shared-b-11d0ca-serving-cert" not found
Jul 09 14:33:03.071180 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:33:03.071118    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f0b32df-989a-49c5-bf26-420af69315f3-maas-api-tls podName:7f0b32df-989a-49c5-bf26-420af69315f3 nodeName:}" failed. No retries permitted until 2026-07-09 14:33:03.571097702 +0000 UTC m=+2215.353472615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7f0b32df-989a-49c5-bf26-420af69315f3-maas-api-tls") pod "maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f" (UID: "7f0b32df-989a-49c5-bf26-420af69315f3") : secret "maas-api-e2e-shared-b-11d0ca-serving-cert" not found
Jul 09 14:33:03.071622 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:03.071603    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f0b32df-989a-49c5-bf26-420af69315f3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f\" (UID: \"7f0b32df-989a-49c5-bf26-420af69315f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"
Jul 09 14:33:03.080464 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:03.080431    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hgfx\" (UniqueName: \"kubernetes.io/projected/7f0b32df-989a-49c5-bf26-420af69315f3-kube-api-access-2hgfx\") pod \"maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f\" (UID: \"7f0b32df-989a-49c5-bf26-420af69315f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"
Jul 09 14:33:03.576269 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:03.576226    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f0b32df-989a-49c5-bf26-420af69315f3-maas-api-tls\") pod \"maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f\" (UID: \"7f0b32df-989a-49c5-bf26-420af69315f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"
Jul 09 14:33:03.578925 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:03.578897    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f0b32df-989a-49c5-bf26-420af69315f3-maas-api-tls\") pod \"maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f\" (UID: \"7f0b32df-989a-49c5-bf26-420af69315f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"
Jul 09 14:33:03.827173 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:03.827089    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"
Jul 09 14:33:03.972786 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:03.972742    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"]
Jul 09 14:33:03.975193 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:33:03.975167    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f0b32df_989a_49c5_bf26_420af69315f3.slice/crio-68e10da805059aa9cd2123d5f33ac9d1686a36b2bd9e8b136edb1585305f86f9 WatchSource:0}: Error finding container 68e10da805059aa9cd2123d5f33ac9d1686a36b2bd9e8b136edb1585305f86f9: Status 404 returned error can't find the container with id 68e10da805059aa9cd2123d5f33ac9d1686a36b2bd9e8b136edb1585305f86f9
Jul 09 14:33:04.695885 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:04.695793    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f" event={"ID":"7f0b32df-989a-49c5-bf26-420af69315f3","Type":"ContainerStarted","Data":"d152e7a5cc0f0f3f6044f468d0921dd61f8c3accbc6317c2ee873af9153e0497"}
Jul 09 14:33:04.695885 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:04.695856    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f" event={"ID":"7f0b32df-989a-49c5-bf26-420af69315f3","Type":"ContainerStarted","Data":"68e10da805059aa9cd2123d5f33ac9d1686a36b2bd9e8b136edb1585305f86f9"}
Jul 09 14:33:04.696129 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:04.695896    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"
Jul 09 14:33:10.706075 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:10.706045    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"
Jul 09 14:33:10.726428 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:33:10.726380    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f" podStartSLOduration=8.287045399 podStartE2EDuration="8.726363909s" totalImagesPullingTime="439.31851ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:33:04.748807883 +0000 UTC m=+2216.531182817" watchObservedRunningTime="2026-07-09 14:33:10.726363909 +0000 UTC m=+2222.508738843"
Jul 09 14:36:08.820948 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:08.820916    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 14:36:08.825471 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:08.825447    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2w5j_9af8ec3e-1ee9-492b-82ed-cdb9b44b3404/ovn-acl-logging/0.log"
Jul 09 14:36:12.898255 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:12.898218    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"]
Jul 09 14:36:12.898738 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:12.898469    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t" podUID="78cbecfa-d7a2-4093-98fc-71fb4c30046f" containerName="maas-api" containerID="cri-o://ff95deac1be2987528e2bb23c48b96a561d22ff8f51ee215a24d82d9fd18c9e6" gracePeriod=30
Jul 09 14:36:13.136120 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.136098    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"
Jul 09 14:36:13.292349 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.292250    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78cbecfa-d7a2-4093-98fc-71fb4c30046f-maas-api-tls\") pod \"78cbecfa-d7a2-4093-98fc-71fb4c30046f\" (UID: \"78cbecfa-d7a2-4093-98fc-71fb4c30046f\") "
Jul 09 14:36:13.292525 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.292360    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlbkm\" (UniqueName: \"kubernetes.io/projected/78cbecfa-d7a2-4093-98fc-71fb4c30046f-kube-api-access-mlbkm\") pod \"78cbecfa-d7a2-4093-98fc-71fb4c30046f\" (UID: \"78cbecfa-d7a2-4093-98fc-71fb4c30046f\") "
Jul 09 14:36:13.292525 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.292394    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78cbecfa-d7a2-4093-98fc-71fb4c30046f-openshift-service-ca\") pod \"78cbecfa-d7a2-4093-98fc-71fb4c30046f\" (UID: \"78cbecfa-d7a2-4093-98fc-71fb4c30046f\") "
Jul 09 14:36:13.292852 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.292820    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78cbecfa-d7a2-4093-98fc-71fb4c30046f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "78cbecfa-d7a2-4093-98fc-71fb4c30046f" (UID: "78cbecfa-d7a2-4093-98fc-71fb4c30046f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:36:13.294585 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.294554    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78cbecfa-d7a2-4093-98fc-71fb4c30046f-kube-api-access-mlbkm" (OuterVolumeSpecName: "kube-api-access-mlbkm") pod "78cbecfa-d7a2-4093-98fc-71fb4c30046f" (UID: "78cbecfa-d7a2-4093-98fc-71fb4c30046f"). InnerVolumeSpecName "kube-api-access-mlbkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:36:13.294700 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.294554    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78cbecfa-d7a2-4093-98fc-71fb4c30046f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "78cbecfa-d7a2-4093-98fc-71fb4c30046f" (UID: "78cbecfa-d7a2-4093-98fc-71fb4c30046f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:36:13.393588 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.393560    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78cbecfa-d7a2-4093-98fc-71fb4c30046f-maas-api-tls\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:36:13.393588 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.393586    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlbkm\" (UniqueName: \"kubernetes.io/projected/78cbecfa-d7a2-4093-98fc-71fb4c30046f-kube-api-access-mlbkm\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:36:13.393787 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.393597    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78cbecfa-d7a2-4093-98fc-71fb4c30046f-openshift-service-ca\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:36:13.483495 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.483456    2579 generic.go:356] "Generic (PLEG): container finished" podID="78cbecfa-d7a2-4093-98fc-71fb4c30046f" containerID="ff95deac1be2987528e2bb23c48b96a561d22ff8f51ee215a24d82d9fd18c9e6" exitCode=0
Jul 09 14:36:13.483661 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.483500    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"
Jul 09 14:36:13.483661 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.483534    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t" event={"ID":"78cbecfa-d7a2-4093-98fc-71fb4c30046f","Type":"ContainerDied","Data":"ff95deac1be2987528e2bb23c48b96a561d22ff8f51ee215a24d82d9fd18c9e6"}
Jul 09 14:36:13.483661 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.483584    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t" event={"ID":"78cbecfa-d7a2-4093-98fc-71fb4c30046f","Type":"ContainerDied","Data":"0bf0ef9d7f1db96e0ea312e142a75b9832e5cfca352e04dd32e642acc3036d48"}
Jul 09 14:36:13.483661 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.483612    2579 scope.go:117] "RemoveContainer" containerID="ff95deac1be2987528e2bb23c48b96a561d22ff8f51ee215a24d82d9fd18c9e6"
Jul 09 14:36:13.494939 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.494914    2579 scope.go:117] "RemoveContainer" containerID="ff95deac1be2987528e2bb23c48b96a561d22ff8f51ee215a24d82d9fd18c9e6"
Jul 09 14:36:13.495404 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:36:13.495377    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff95deac1be2987528e2bb23c48b96a561d22ff8f51ee215a24d82d9fd18c9e6\": container with ID starting with ff95deac1be2987528e2bb23c48b96a561d22ff8f51ee215a24d82d9fd18c9e6 not found: ID does not exist" containerID="ff95deac1be2987528e2bb23c48b96a561d22ff8f51ee215a24d82d9fd18c9e6"
Jul 09 14:36:13.495497 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.495411    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff95deac1be2987528e2bb23c48b96a561d22ff8f51ee215a24d82d9fd18c9e6"} err="failed to get container status \"ff95deac1be2987528e2bb23c48b96a561d22ff8f51ee215a24d82d9fd18c9e6\": rpc error: code = NotFound desc = could not find container \"ff95deac1be2987528e2bb23c48b96a561d22ff8f51ee215a24d82d9fd18c9e6\": container with ID starting with ff95deac1be2987528e2bb23c48b96a561d22ff8f51ee215a24d82d9fd18c9e6 not found: ID does not exist"
Jul 09 14:36:13.505141 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.505117    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"]
Jul 09 14:36:13.509822 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:13.509802    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6b90ba-7f98fd67fd-g6c6t"]
Jul 09 14:36:14.824874 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:14.824842    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78cbecfa-d7a2-4093-98fc-71fb4c30046f" path="/var/lib/kubelet/pods/78cbecfa-d7a2-4093-98fc-71fb4c30046f/volumes"
Jul 09 14:36:20.025290 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.025244    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"]
Jul 09 14:36:20.025731 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.025491    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f" podUID="7f0b32df-989a-49c5-bf26-420af69315f3" containerName="maas-api" containerID="cri-o://d152e7a5cc0f0f3f6044f468d0921dd61f8c3accbc6317c2ee873af9153e0497" gracePeriod=30
Jul 09 14:36:20.275095 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.275071    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"
Jul 09 14:36:20.355884 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.355799    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f0b32df-989a-49c5-bf26-420af69315f3-maas-api-tls\") pod \"7f0b32df-989a-49c5-bf26-420af69315f3\" (UID: \"7f0b32df-989a-49c5-bf26-420af69315f3\") "
Jul 09 14:36:20.356039 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.355921    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hgfx\" (UniqueName: \"kubernetes.io/projected/7f0b32df-989a-49c5-bf26-420af69315f3-kube-api-access-2hgfx\") pod \"7f0b32df-989a-49c5-bf26-420af69315f3\" (UID: \"7f0b32df-989a-49c5-bf26-420af69315f3\") "
Jul 09 14:36:20.356039 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.355962    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f0b32df-989a-49c5-bf26-420af69315f3-openshift-service-ca\") pod \"7f0b32df-989a-49c5-bf26-420af69315f3\" (UID: \"7f0b32df-989a-49c5-bf26-420af69315f3\") "
Jul 09 14:36:20.356355 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.356331    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f0b32df-989a-49c5-bf26-420af69315f3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7f0b32df-989a-49c5-bf26-420af69315f3" (UID: "7f0b32df-989a-49c5-bf26-420af69315f3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:36:20.358035 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.358007    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f0b32df-989a-49c5-bf26-420af69315f3-kube-api-access-2hgfx" (OuterVolumeSpecName: "kube-api-access-2hgfx") pod "7f0b32df-989a-49c5-bf26-420af69315f3" (UID: "7f0b32df-989a-49c5-bf26-420af69315f3"). InnerVolumeSpecName "kube-api-access-2hgfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:36:20.358035 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.358018    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f0b32df-989a-49c5-bf26-420af69315f3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7f0b32df-989a-49c5-bf26-420af69315f3" (UID: "7f0b32df-989a-49c5-bf26-420af69315f3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:36:20.457341 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.457319    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f0b32df-989a-49c5-bf26-420af69315f3-maas-api-tls\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:36:20.457341 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.457340    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hgfx\" (UniqueName: \"kubernetes.io/projected/7f0b32df-989a-49c5-bf26-420af69315f3-kube-api-access-2hgfx\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:36:20.457518 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.457349    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f0b32df-989a-49c5-bf26-420af69315f3-openshift-service-ca\") on node \"ip-10-0-128-186.ec2.internal\" DevicePath \"\""
Jul 09 14:36:20.515300 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.515265    2579 generic.go:356] "Generic (PLEG): container finished" podID="7f0b32df-989a-49c5-bf26-420af69315f3" containerID="d152e7a5cc0f0f3f6044f468d0921dd61f8c3accbc6317c2ee873af9153e0497" exitCode=0
Jul 09 14:36:20.515440 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.515318    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"
Jul 09 14:36:20.515440 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.515348    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f" event={"ID":"7f0b32df-989a-49c5-bf26-420af69315f3","Type":"ContainerDied","Data":"d152e7a5cc0f0f3f6044f468d0921dd61f8c3accbc6317c2ee873af9153e0497"}
Jul 09 14:36:20.515440 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.515403    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f" event={"ID":"7f0b32df-989a-49c5-bf26-420af69315f3","Type":"ContainerDied","Data":"68e10da805059aa9cd2123d5f33ac9d1686a36b2bd9e8b136edb1585305f86f9"}
Jul 09 14:36:20.515440 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.515426    2579 scope.go:117] "RemoveContainer" containerID="d152e7a5cc0f0f3f6044f468d0921dd61f8c3accbc6317c2ee873af9153e0497"
Jul 09 14:36:20.523885 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.523867    2579 scope.go:117] "RemoveContainer" containerID="d152e7a5cc0f0f3f6044f468d0921dd61f8c3accbc6317c2ee873af9153e0497"
Jul 09 14:36:20.524156 ip-10-0-128-186 kubenswrapper[2579]: E0709 14:36:20.524138    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d152e7a5cc0f0f3f6044f468d0921dd61f8c3accbc6317c2ee873af9153e0497\": container with ID starting with d152e7a5cc0f0f3f6044f468d0921dd61f8c3accbc6317c2ee873af9153e0497 not found: ID does not exist" containerID="d152e7a5cc0f0f3f6044f468d0921dd61f8c3accbc6317c2ee873af9153e0497"
Jul 09 14:36:20.524240 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.524165    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d152e7a5cc0f0f3f6044f468d0921dd61f8c3accbc6317c2ee873af9153e0497"} err="failed to get container status \"d152e7a5cc0f0f3f6044f468d0921dd61f8c3accbc6317c2ee873af9153e0497\": rpc error: code = NotFound desc = could not find container \"d152e7a5cc0f0f3f6044f468d0921dd61f8c3accbc6317c2ee873af9153e0497\": container with ID starting with d152e7a5cc0f0f3f6044f468d0921dd61f8c3accbc6317c2ee873af9153e0497 not found: ID does not exist"
Jul 09 14:36:20.547709 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.547682    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"]
Jul 09 14:36:20.551506 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.551475    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-11d0ca-55566f8b5c-c2d4f"]
Jul 09 14:36:20.824835 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:20.824806    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f0b32df-989a-49c5-bf26-420af69315f3" path="/var/lib/kubelet/pods/7f0b32df-989a-49c5-bf26-420af69315f3/volumes"
Jul 09 14:36:32.155680 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:32.155605    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d74b946fd-qqm4r_9bc7d447-b080-4a56-a248-9419f4bef98f/copy-manifests/0.log"
Jul 09 14:36:32.168613 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:32.168586    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d74b946fd-qqm4r_9bc7d447-b080-4a56-a248-9419f4bef98f/manager/0.log"
Jul 09 14:36:35.090999 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:35.090972    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f974f5cb9-xb4qw_185a4d54-6cfd-4e0c-a5b6-36c68f8389e5/kube-auth-proxy/0.log"
Jul 09 14:36:35.899353 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:35.899326    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-7j6wv_87d2e702-2781-4a35-9d44-778c95f312d0/main/0.log"
Jul 09 14:36:36.347679 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:36.347599    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-ddm4g_05726ee1-9457-4a26-965c-4fcc8d6b8681/main/0.log"
Jul 09 14:36:36.454735 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:36.454701    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-mzd7h_685d7301-539d-4cb1-b2ac-d77ae501b6ba/main/0.log"
Jul 09 14:36:40.237793 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.237762    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fq7xw/must-gather-cvflh"]
Jul 09 14:36:40.238172 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.238042    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78cbecfa-d7a2-4093-98fc-71fb4c30046f" containerName="maas-api"
Jul 09 14:36:40.238172 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.238052    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="78cbecfa-d7a2-4093-98fc-71fb4c30046f" containerName="maas-api"
Jul 09 14:36:40.238172 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.238071    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f0b32df-989a-49c5-bf26-420af69315f3" containerName="maas-api"
Jul 09 14:36:40.238172 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.238076    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f0b32df-989a-49c5-bf26-420af69315f3" containerName="maas-api"
Jul 09 14:36:40.238172 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.238131    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="78cbecfa-d7a2-4093-98fc-71fb4c30046f" containerName="maas-api"
Jul 09 14:36:40.238172 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.238140    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f0b32df-989a-49c5-bf26-420af69315f3" containerName="maas-api"
Jul 09 14:36:40.242090 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.242072    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fq7xw/must-gather-cvflh"
Jul 09 14:36:40.244721 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.244695    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fq7xw\"/\"kube-root-ca.crt\""
Jul 09 14:36:40.244836 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.244695    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fq7xw\"/\"default-dockercfg-vd84f\""
Jul 09 14:36:40.245652 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.245632    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fq7xw\"/\"openshift-service-ca.crt\""
Jul 09 14:36:40.249724 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.249698    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fq7xw/must-gather-cvflh"]
Jul 09 14:36:40.326796 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.326769    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55nl7\" (UniqueName: \"kubernetes.io/projected/a70b88a5-03ea-4449-8e5e-0980ab210c94-kube-api-access-55nl7\") pod \"must-gather-cvflh\" (UID: \"a70b88a5-03ea-4449-8e5e-0980ab210c94\") " pod="openshift-must-gather-fq7xw/must-gather-cvflh"
Jul 09 14:36:40.326925 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.326801    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a70b88a5-03ea-4449-8e5e-0980ab210c94-must-gather-output\") pod \"must-gather-cvflh\" (UID: \"a70b88a5-03ea-4449-8e5e-0980ab210c94\") " pod="openshift-must-gather-fq7xw/must-gather-cvflh"
Jul 09 14:36:40.428182 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.428148    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55nl7\" (UniqueName: \"kubernetes.io/projected/a70b88a5-03ea-4449-8e5e-0980ab210c94-kube-api-access-55nl7\") pod \"must-gather-cvflh\" (UID: \"a70b88a5-03ea-4449-8e5e-0980ab210c94\") " pod="openshift-must-gather-fq7xw/must-gather-cvflh"
Jul 09 14:36:40.428182 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.428183    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a70b88a5-03ea-4449-8e5e-0980ab210c94-must-gather-output\") pod \"must-gather-cvflh\" (UID: \"a70b88a5-03ea-4449-8e5e-0980ab210c94\") " pod="openshift-must-gather-fq7xw/must-gather-cvflh"
Jul 09 14:36:40.428501 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.428472    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a70b88a5-03ea-4449-8e5e-0980ab210c94-must-gather-output\") pod \"must-gather-cvflh\" (UID: \"a70b88a5-03ea-4449-8e5e-0980ab210c94\") " pod="openshift-must-gather-fq7xw/must-gather-cvflh"
Jul 09 14:36:40.437697 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.437669    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55nl7\" (UniqueName: \"kubernetes.io/projected/a70b88a5-03ea-4449-8e5e-0980ab210c94-kube-api-access-55nl7\") pod \"must-gather-cvflh\" (UID: \"a70b88a5-03ea-4449-8e5e-0980ab210c94\") " pod="openshift-must-gather-fq7xw/must-gather-cvflh"
Jul 09 14:36:40.551896 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.551806    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fq7xw/must-gather-cvflh"
Jul 09 14:36:40.683803 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.683636    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fq7xw/must-gather-cvflh"]
Jul 09 14:36:40.686537 ip-10-0-128-186 kubenswrapper[2579]: W0709 14:36:40.686513    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda70b88a5_03ea_4449_8e5e_0980ab210c94.slice/crio-be4beb6da310aafa8fc7ff89106a7e37ebccca13dcb91717208e24c4c1290fdb WatchSource:0}: Error finding container be4beb6da310aafa8fc7ff89106a7e37ebccca13dcb91717208e24c4c1290fdb: Status 404 returned error can't find the container with id be4beb6da310aafa8fc7ff89106a7e37ebccca13dcb91717208e24c4c1290fdb
Jul 09 14:36:40.688741 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:40.688723    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 14:36:41.603411 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:41.603384    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fq7xw/must-gather-cvflh" event={"ID":"a70b88a5-03ea-4449-8e5e-0980ab210c94","Type":"ContainerStarted","Data":"be4beb6da310aafa8fc7ff89106a7e37ebccca13dcb91717208e24c4c1290fdb"}
Jul 09 14:36:42.611661 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:42.611615    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fq7xw/must-gather-cvflh" event={"ID":"a70b88a5-03ea-4449-8e5e-0980ab210c94","Type":"ContainerStarted","Data":"b010b8df1cdf844e17501279da7bbf80f1099c2d7d7dfb441cad3b39af835768"}
Jul 09 14:36:42.612160 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:42.611674    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fq7xw/must-gather-cvflh" event={"ID":"a70b88a5-03ea-4449-8e5e-0980ab210c94","Type":"ContainerStarted","Data":"19f5fd7c13c326b8e3c7d138889e0c4d7dfb867ba3f582e20475d81c94a0c1e7"}
Jul 09 14:36:43.265071 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:43.265031    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sv4nr_fb88988f-93b8-458f-8e2a-19c0ec6db469/global-pull-secret-syncer/0.log"
Jul 09 14:36:43.474033 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:43.474004    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jhxx7_12db20ba-2e42-4777-9558-d4f6ce9587e9/konnectivity-agent/0.log"
Jul 09 14:36:43.499885 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:43.499856    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-186.ec2.internal_92fbe84b040827f8e74fa00dccfc7dbd/haproxy/0.log"
Jul 09 14:36:51.350259 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:51.350167    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6999bd6c8b-bc7bj_3fa17005-8c04-4cda-a6ff-475400d5a6a0/metrics-server/0.log"
Jul 09 14:36:51.610826 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:51.610166    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrfzs_972019b0-de63-44ce-86c1-8faaa019ef6d/node-exporter/0.log"
Jul 09 14:36:51.637084 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:51.637035    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrfzs_972019b0-de63-44ce-86c1-8faaa019ef6d/kube-rbac-proxy/0.log"
Jul 09 14:36:51.660944 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:51.660915    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrfzs_972019b0-de63-44ce-86c1-8faaa019ef6d/init-textfile/0.log"
Jul 09 14:36:51.785627 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:51.785599    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f82c8899-47f3-4f37-9828-4410ee20c388/prometheus/0.log"
Jul 09 14:36:51.810483 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:51.810448    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f82c8899-47f3-4f37-9828-4410ee20c388/config-reloader/0.log"
Jul 09 14:36:51.840178 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:51.840142    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f82c8899-47f3-4f37-9828-4410ee20c388/thanos-sidecar/0.log"
Jul 09 14:36:51.865096 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:51.865022    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f82c8899-47f3-4f37-9828-4410ee20c388/kube-rbac-proxy-web/0.log"
Jul 09 14:36:51.890055 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:51.890026    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f82c8899-47f3-4f37-9828-4410ee20c388/kube-rbac-proxy/0.log"
Jul 09 14:36:51.917142 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:51.917112    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f82c8899-47f3-4f37-9828-4410ee20c388/kube-rbac-proxy-thanos/0.log"
Jul 09 14:36:51.944781 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:51.944714    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f82c8899-47f3-4f37-9828-4410ee20c388/init-config-reloader/0.log"
Jul 09 14:36:52.043510 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:52.043479    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-b6ctd_f85b5eb2-6cae-4366-a270-c3478df155bc/prometheus-operator-admission-webhook/0.log"
Jul 09 14:36:53.054713 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.054646    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fq7xw/must-gather-cvflh" podStartSLOduration=12.218928486 podStartE2EDuration="13.054623216s" totalImagesPullingTime="835.69473ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:36:42.631018103 +0000 UTC m=+2434.413393038" watchObservedRunningTime="2026-07-09 14:36:53.054623216 +0000 UTC m=+2444.836998150"
Jul 09 14:36:53.055684 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.055655    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"]
Jul 09 14:36:53.061780 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.061733    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.066670 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.066636    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"]
Jul 09 14:36:53.167527 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.167489    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdn2r\" (UniqueName: \"kubernetes.io/projected/537782a6-07a0-41d6-ab7c-6f4280525666-kube-api-access-zdn2r\") pod \"perf-node-gather-daemonset-6wskj\" (UID: \"537782a6-07a0-41d6-ab7c-6f4280525666\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.167729 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.167548    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/537782a6-07a0-41d6-ab7c-6f4280525666-proc\") pod \"perf-node-gather-daemonset-6wskj\" (UID: \"537782a6-07a0-41d6-ab7c-6f4280525666\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.167729 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.167578    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/537782a6-07a0-41d6-ab7c-6f4280525666-podres\") pod \"perf-node-gather-daemonset-6wskj\" (UID: \"537782a6-07a0-41d6-ab7c-6f4280525666\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.167729 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.167606    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/537782a6-07a0-41d6-ab7c-6f4280525666-sys\") pod \"perf-node-gather-daemonset-6wskj\" (UID: \"537782a6-07a0-41d6-ab7c-6f4280525666\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.167729 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.167627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/537782a6-07a0-41d6-ab7c-6f4280525666-lib-modules\") pod \"perf-node-gather-daemonset-6wskj\" (UID: \"537782a6-07a0-41d6-ab7c-6f4280525666\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.268323 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.268286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/537782a6-07a0-41d6-ab7c-6f4280525666-proc\") pod \"perf-node-gather-daemonset-6wskj\" (UID: \"537782a6-07a0-41d6-ab7c-6f4280525666\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.268515 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.268344    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/537782a6-07a0-41d6-ab7c-6f4280525666-podres\") pod \"perf-node-gather-daemonset-6wskj\" (UID: \"537782a6-07a0-41d6-ab7c-6f4280525666\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.268515 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.268458    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/537782a6-07a0-41d6-ab7c-6f4280525666-proc\") pod \"perf-node-gather-daemonset-6wskj\" (UID: \"537782a6-07a0-41d6-ab7c-6f4280525666\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.268515 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.268504    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/537782a6-07a0-41d6-ab7c-6f4280525666-podres\") pod \"perf-node-gather-daemonset-6wskj\" (UID: \"537782a6-07a0-41d6-ab7c-6f4280525666\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.268682 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.268532    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/537782a6-07a0-41d6-ab7c-6f4280525666-sys\") pod \"perf-node-gather-daemonset-6wskj\" (UID: \"537782a6-07a0-41d6-ab7c-6f4280525666\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.268682 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.268585    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/537782a6-07a0-41d6-ab7c-6f4280525666-lib-modules\") pod \"perf-node-gather-daemonset-6wskj\" (UID: \"537782a6-07a0-41d6-ab7c-6f4280525666\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.268682 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.268622    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/537782a6-07a0-41d6-ab7c-6f4280525666-sys\") pod \"perf-node-gather-daemonset-6wskj\" (UID: \"537782a6-07a0-41d6-ab7c-6f4280525666\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.268682 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.268636    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdn2r\" (UniqueName: \"kubernetes.io/projected/537782a6-07a0-41d6-ab7c-6f4280525666-kube-api-access-zdn2r\") pod \"perf-node-gather-daemonset-6wskj\" (UID: \"537782a6-07a0-41d6-ab7c-6f4280525666\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.268913 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.268737    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/537782a6-07a0-41d6-ab7c-6f4280525666-lib-modules\") pod \"perf-node-gather-daemonset-6wskj\" (UID: \"537782a6-07a0-41d6-ab7c-6f4280525666\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.279373 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.279340    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdn2r\" (UniqueName: \"kubernetes.io/projected/537782a6-07a0-41d6-ab7c-6f4280525666-kube-api-access-zdn2r\") pod \"perf-node-gather-daemonset-6wskj\" (UID: \"537782a6-07a0-41d6-ab7c-6f4280525666\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.374411 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.374328    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:53.548427 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.548345    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"]
Jul 09 14:36:53.682771 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:53.682720    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj" event={"ID":"537782a6-07a0-41d6-ab7c-6f4280525666","Type":"ContainerStarted","Data":"9368926ca88d3e3e030e79cca61d533428896724550f9fd6c8d340f1e4fe5cce"}
Jul 09 14:36:54.690617 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:54.690575    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj" event={"ID":"537782a6-07a0-41d6-ab7c-6f4280525666","Type":"ContainerStarted","Data":"a41f94b7c55f3a151dad8dedb42c1eb635b1ff0385ee2f152f3d8a55fbb95c39"}
Jul 09 14:36:54.691483 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:54.691460    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:36:54.711043 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:54.710979    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj" podStartSLOduration=1.710959992 podStartE2EDuration="1.710959992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:36:54.708257186 +0000 UTC m=+2446.490632120" watchObservedRunningTime="2026-07-09 14:36:54.710959992 +0000 UTC m=+2446.493334925"
Jul 09 14:36:56.290242 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:56.290212    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jnpml_7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9/dns/0.log"
Jul 09 14:36:56.317212 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:56.317186    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jnpml_7d0c72fa-1d31-4aa6-b9a4-e1f237fbdfd9/kube-rbac-proxy/0.log"
Jul 09 14:36:56.429184 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:56.429148    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wr822_950143f5-da10-4fa7-897f-cd547c9932d0/dns-node-resolver/0.log"
Jul 09 14:36:57.068799 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:57.068765    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8rxdv_b39fdbde-6632-4711-a42a-67a86d708c58/node-ca/0.log"
Jul 09 14:36:58.295799 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:58.295773    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f974f5cb9-xb4qw_185a4d54-6cfd-4e0c-a5b6-36c68f8389e5/kube-auth-proxy/0.log"
Jul 09 14:36:59.174613 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:59.174578    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-456t7_41620e9f-76f0-410a-8b47-d1a33dca6722/serve-healthcheck-canary/0.log"
Jul 09 14:36:59.909561 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:59.909514    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pwth7_aaf6e0b7-e1c1-41d0-805a-622653b89228/kube-rbac-proxy/0.log"
Jul 09 14:36:59.935792 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:59.935760    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pwth7_aaf6e0b7-e1c1-41d0-805a-622653b89228/exporter/0.log"
Jul 09 14:36:59.960342 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:36:59.960313    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pwth7_aaf6e0b7-e1c1-41d0-805a-622653b89228/extractor/0.log"
Jul 09 14:37:01.716563 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:37:01.716527    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-6wskj"
Jul 09 14:37:02.592916 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:37:02.592877    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d74b946fd-qqm4r_9bc7d447-b080-4a56-a248-9419f4bef98f/manager/0.log"
Jul 09 14:37:02.616227 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:37:02.616198    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d74b946fd-qqm4r_9bc7d447-b080-4a56-a248-9419f4bef98f/copy-manifests/0.log"
Jul 09 14:37:03.974983 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:37:03.974872    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f974f5cb9-xb4qw_185a4d54-6cfd-4e0c-a5b6-36c68f8389e5/kube-auth-proxy/0.log"
Jul 09 14:37:04.071173 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:37:04.071101    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-c98bd85cf-j8b8f_943e2729-26ed-4bd1-a94c-ca9ce3b01cf7/manager/0.log"
Jul 09 14:37:05.537835 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:37:05.537789    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f974f5cb9-xb4qw_185a4d54-6cfd-4e0c-a5b6-36c68f8389e5/kube-auth-proxy/0.log"
Jul 09 14:37:11.054667 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:37:11.054596    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mpgcc_e9039e54-a69f-4385-96ea-25d6ffa237c0/kube-multus-additional-cni-plugins/0.log"
Jul 09 14:37:11.078969 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:37:11.078938    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mpgcc_e9039e54-a69f-4385-96ea-25d6ffa237c0/egress-router-binary-copy/0.log"
Jul 09 14:37:11.105676 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:37:11.105641    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mpgcc_e9039e54-a69f-4385-96ea-25d6ffa237c0/cni-plugins/0.log"
Jul 09 14:37:11.129432 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:37:11.129404    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mpgcc_e9039e54-a69f-4385-96ea-25d6ffa237c0/bond-cni-plugin/0.log"
Jul 09 14:37:11.153632 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:37:11.153606    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mpgcc_e9039e54-a69f-4385-96ea-25d6ffa237c0/routeoverride-cni/0.log"
Jul 09 14:37:11.176736 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:37:11.176691    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mpgcc_e9039e54-a69f-4385-96ea-25d6ffa237c0/whereabouts-cni-bincopy/0.log"
Jul 09 14:37:11.199680 ip-10-0-128-186 kubenswrapper[2579]: I0709 14:37:11.199651    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mpgcc_e9039e54-a69f-4385-96ea-25d6ffa237c0/whereabouts-cni/0.log"