Jul 29 18:32:23.362868 ip-10-0-137-187 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 29 18:32:23.362881 ip-10-0-137-187 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 29 18:32:23.362889 ip-10-0-137-187 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 29 18:32:23.363284 ip-10-0-137-187 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 29 18:32:34.599485 ip-10-0-137-187 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 29 18:32:34.599498 ip-10-0-137-187 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 1b876685fd91487bb238990d0a91425e --
Jul 29 18:34:33.938777 ip-10-0-137-187 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 18:34:34.319851 ip-10-0-137-187 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 18:34:34.319851 ip-10-0-137-187 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 18:34:34.319851 ip-10-0-137-187 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 18:34:34.319851 ip-10-0-137-187 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 18:34:34.319851 ip-10-0-137-187 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 18:34:34.320853 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.320770    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 18:34:34.324635 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324619    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 18:34:34.324635 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324635    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324638    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324643    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324646    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324649    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324653    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324664    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324668    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324670    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324674    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324676    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324679    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324682    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324684    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324687    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324696    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324698    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324701    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324704    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 18:34:34.324703 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324706    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324709    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324712    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324715    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324717    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324720    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324722    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324725    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324727    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324730    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324732    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324734    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324737    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324739    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324741    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324743    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324746    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324748    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324751    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324754    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 18:34:34.325120 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324757    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324759    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324761    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324764    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324767    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324769    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324771    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324774    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324776    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324778    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324781    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324783    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324785    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324787    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324790    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324793    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324795    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324798    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324801    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324803    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 18:34:34.325618 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324805    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324808    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324810    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324813    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324816    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324820    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324823    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324825    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324828    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324831    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324833    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324840    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324843    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324846    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324848    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324850    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324853    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324855    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324858    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 18:34:34.326070 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324860    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324862    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324864    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324867    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324869    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324872    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324875    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324879    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324883    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324886    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324889    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324892    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324894    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324897    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324900    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324903    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324905    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324908    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324911    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.324913    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 18:34:34.326512 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325594    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325600    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325603    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325606    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325608    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325614    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325617    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325620    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325623    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325625    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325628    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325630    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325632    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325635    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325637    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325639    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325642    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325644    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325648    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325651    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 18:34:34.326970 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325653    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325656    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325660    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325664    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325666    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325669    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325672    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325674    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325676    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325679    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325682    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325685    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325688    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325690    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325693    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325695    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325698    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325700    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325702    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 18:34:34.327415 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325705    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325707    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325709    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325712    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325714    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325718    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325721    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325723    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325726    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325729    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325731    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325734    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325737    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325739    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325741    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325744    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325746    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325749    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325751    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 18:34:34.327897 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325753    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325756    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325758    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325761    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325763    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325765    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325768    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325770    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325772    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325775    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325778    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325781    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325785    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325788    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325791    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325793    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325795    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325798    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325800    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325802    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 18:34:34.328319 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325804    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325807    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325809    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325812    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325814    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325817    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325819    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325821    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325824    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325826    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325828    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325831    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325833    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325835    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325838    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325840    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325843    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325845    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325847    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325850    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 18:34:34.328800 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.325852    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326196    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326200    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326203    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326206    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326209    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326212    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326214    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326216    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326219    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326221    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326223    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326226    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326228    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326231    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326233    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326236    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326239    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326241    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326244    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 18:34:34.329242 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326246    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326249    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326251    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326254    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326256    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326259    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326261    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326263    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326266    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326268    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326271    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326273    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326276    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326278    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326281    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326283    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326285    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326288    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326290    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326293    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 18:34:34.329720 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326295    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326297    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326300    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326304    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326307    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326309    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326312    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326315    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326317    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326321    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326324    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326327    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326329    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326332    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326334    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326337    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326340    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326342    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326344    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 18:34:34.330174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326347    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326349    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326351    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326353    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326359    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326361    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326364    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326366    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326368    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326371    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326373    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326376    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326379    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326381    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326383    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326386    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326388    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326390    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326393    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326395    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 18:34:34.330640 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326397    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326400    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326402    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326404    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326407    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326409    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326412    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326414    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326417    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326426    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326429    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326431    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326435    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326438    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326441    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326443    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326447    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326449    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326452    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326454    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 18:34:34.331094 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.326457    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326541    2579 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326552    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326559    2579 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326563    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326567    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326570    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326574    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326578    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326581    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326584    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326587    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326590    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326593    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326596    2579 flags.go:64] FLAG: --cgroup-root=""
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326599    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326602    2579 flags.go:64] FLAG: --client-ca-file=""
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326604    2579 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326607    2579 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326611    2579 flags.go:64] FLAG: --cluster-domain=""
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326614    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326617    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326620    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326623    2579 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 18:34:34.331573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326626    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326629    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326632    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326635    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326639    2579 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326645    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326648    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326651    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326654    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326658    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326661    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326664    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326667    2579 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326669    2579 flags.go:64] FLAG: --enable-server="true"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326673    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326677    2579 flags.go:64] FLAG: --event-burst="100"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326680    2579 flags.go:64] FLAG: --event-qps="50"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326683    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326686    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326689    2579 flags.go:64] FLAG: --eviction-hard=""
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326692    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326695    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326698    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326701    2579 flags.go:64] FLAG: --eviction-soft=""
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326703    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 18:34:34.332109 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326706    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326708    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326711    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326714    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326717    2579 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326719    2579 flags.go:64] FLAG: --feature-gates=""
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326723    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326725    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326728    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326731    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326734    2579 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326738    2579 flags.go:64] FLAG: --help="false"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326742    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-137-187.ec2.internal"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326745    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326748    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326751    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326754    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326757    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326760    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326762    2579 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326765    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326768    2579 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326771    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326774    2579 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 18:34:34.332740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326777    2579 flags.go:64] FLAG: --kube-reserved=""
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326780    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326782    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326785    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326788    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326790    2579 flags.go:64] FLAG: --lock-file=""
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326793    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326796    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326798    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326803    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326806    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326809    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326811    2579 flags.go:64] FLAG: --logging-format="text"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326814    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326817    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326820    2579 flags.go:64] FLAG: --manifest-url=""
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326822    2579 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326826    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326829    2579 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326834    2579 flags.go:64] FLAG: --max-pods="110"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326838    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326841    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326843    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326846    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326849    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 18:34:34.333384 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326852    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326855    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326861    2579 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326864    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326867    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326870    2579 flags.go:64] FLAG: --pod-cidr=""
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326873    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326878    2579 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326881    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326884    2579 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326886    2579 flags.go:64] FLAG: --port="10250"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326889    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326892    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03a0f1f2213b7745a"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326895    2579 flags.go:64] FLAG: --qos-reserved=""
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326897    2579 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326900    2579 flags.go:64] FLAG: --register-node="true"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326903    2579 flags.go:64] FLAG: --register-with-taints=""
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326906    2579 flags.go:64] FLAG: --registry-burst="10"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326909    2579 flags.go:64] FLAG: --registry-qps="5"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326911    2579 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326914    2579 flags.go:64] FLAG: --reserved-memory=""
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326918    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326920    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326923    2579 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326926    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 18:34:34.334019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326929    2579 flags.go:64] FLAG: --runonce="false"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326932    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326935    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326939    2579 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326942    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326945    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326948    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326951    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326954    2579 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326956    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326959    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326962    2579 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326965    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326968    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326971    2579 flags.go:64] FLAG: --system-cgroups=""
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326974    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326979    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326981    2579 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326984    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326988    2579 flags.go:64] FLAG: --tls-min-version=""
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326990    2579 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326993    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326996    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.326998    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.327001    2579 flags.go:64] FLAG: --v="2"
Jul 29 18:34:34.334600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.327004    2579 flags.go:64] FLAG: --version="false"
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.327008    2579 flags.go:64] FLAG: --vmodule=""
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.327014    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.327018    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327121    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327125    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327128    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327131    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327135    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327138    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327142    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327145    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327147    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327151    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327154    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327157    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327159    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327162    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327165    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 18:34:34.335149 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327168    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327170    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327173    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327176    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327179    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327182    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327184    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327187    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327189    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327192    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327194    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327196    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327199    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327201    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327203    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327206    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327208    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327210    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327213    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 18:34:34.335596 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327215    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327217    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327221    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327223    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327227    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327229    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327232    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327234    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327236    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327239    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327241    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327243    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327246    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327248    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327251    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327253    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327255    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327258    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327260    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327263    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 18:34:34.336028 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327265    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327268    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327270    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327273    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327275    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327277    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327280    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327282    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327284    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327287    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327289    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327292    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327294    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327297    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327302    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327305    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327309    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327312    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327314    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327317    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 18:34:34.336530 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327319    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327322    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327326    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327329    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327331    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327334    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327336    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327339    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327341    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327344    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327346    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327349    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327351    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327354    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327356    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327358    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327361    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327363    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327365    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327368    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 18:34:34.336977 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327370    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 18:34:34.337445 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327372    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 18:34:34.337445 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327375    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 18:34:34.337445 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327377    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 18:34:34.337445 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.327379    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 18:34:34.337445 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.327919    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 18:34:34.337445 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.334015    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 18:34:34.337445 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.334031    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 18:34:34.337445 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.334826    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 18:34:34.337445 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.334834    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 18:34:34.337445 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334892    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 18:34:34.337445 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334898    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 18:34:34.337445 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334901    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 18:34:34.337445 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334905    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 18:34:34.337445 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334908    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 18:34:34.337445 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334910    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334913    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334915    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334918    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334920    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334923    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334925    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334928    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334930    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334932    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334935    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334937    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334940    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334942    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334944    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334947    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334950    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334952    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334955    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334957    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 18:34:34.337805 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334960    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334962    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334965    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334967    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334975    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334978    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334980    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334983    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334986    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334990    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334993    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334996    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.334999    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335001    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335004    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335006    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335008    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335011    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335013    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 18:34:34.338252 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335016    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335018    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335020    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335023    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335025    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335028    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335030    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335032    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335035    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335037    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335041    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335045    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335047    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335050    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335052    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335054    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335056    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335059    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335062    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 18:34:34.338690 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335065    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335068    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335071    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335073    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335075    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335078    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335080    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335084    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335087    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335090    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335093    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335096    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335098    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335100    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335103    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335105    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335108    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335110    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335113    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335115    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 18:34:34.339114 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335117    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 18:34:34.339620 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335120    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 18:34:34.339620 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335122    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 18:34:34.339620 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335125    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 18:34:34.339620 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335127    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 18:34:34.339620 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335129    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 18:34:34.339620 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335133    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 18:34:34.339620 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335136    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 18:34:34.339620 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335138    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 18:34:34.339620 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335141    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 18:34:34.339620 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335143    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 18:34:34.339620 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335146    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 18:34:34.339620 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335149    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 18:34:34.339620 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335151    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 18:34:34.339620 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335154    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 18:34:34.339620 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335157    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 18:34:34.339944 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.335162    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 18:34:34.339944 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335270    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 18:34:34.339944 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335275    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 18:34:34.339944 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335278    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 18:34:34.339944 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335280    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 18:34:34.339944 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335283    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 18:34:34.339944 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335285    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 18:34:34.339944 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335289    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 18:34:34.339944 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335293    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 18:34:34.339944 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335295    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 18:34:34.339944 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335298    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 18:34:34.339944 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335301    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 18:34:34.339944 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335303    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 18:34:34.339944 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335305    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 18:34:34.339944 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335308    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335310    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335312    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335315    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335317    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335320    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335322    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335324    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335327    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335330    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335332    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335335    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335337    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335339    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335343    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335346    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335349    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335352    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335355    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335358    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 18:34:34.340283 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335360    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335362    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335365    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335367    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335369    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335372    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335374    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335376    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335378    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335381    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335383    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335385    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335387    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335390    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335392    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335394    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335398    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335400    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335402    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335405    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 18:34:34.340741 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335407    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335410    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335412    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335414    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335417    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335424    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335426    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335429    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335431    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335434    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335436    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335440    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335443    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335446    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335448    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335451    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335454    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335456    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335459    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335461    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 18:34:34.341174 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335463    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335466    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335468    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335470    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335473    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335475    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335478    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335480    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335483    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335485    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335487    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335490    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335492    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335499    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335510    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335513    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335515    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335517    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335520    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 18:34:34.341680 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335522    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 18:34:34.342134 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335524    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 18:34:34.342134 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335527    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 18:34:34.342134 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335529    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 18:34:34.342134 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335532    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 18:34:34.342134 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335534    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 18:34:34.342134 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:34.335536    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 18:34:34.342134 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.335541    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 18:34:34.342134 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.335659    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 18:34:34.342134 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.337550    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 18:34:34.342134 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.338698    2579 server.go:1019] "Starting client certificate rotation"
Jul 29 18:34:34.342134 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.338803    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 18:34:34.342134 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.338863    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 18:34:34.342134 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.341315    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 18:34:34.345344 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.345321    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 18:34:34.353768 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.353753    2579 log.go:25] "Validated CRI v1 runtime API"
Jul 29 18:34:34.355586 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.355572    2579 log.go:25] "Validated CRI v1 image API"
Jul 29 18:34:34.358584 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.358567    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 18:34:34.361725 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.361706    2579 fs.go:135] Filesystem UUIDs: map[5f3fba25-917f-461b-b2a3-e8de6ece8236:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c2683d4f-7728-429a-8c46-9d935abefddc:/dev/nvme0n1p3]
Jul 29 18:34:34.361768 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.361725    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 18:34:34.365660 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.365570    2579 manager.go:216] Machine: {Timestamp:2026-07-29 18:34:34.365105637 +0000 UTC m=+0.316672702 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100120 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e9d552116a3a5c39b4fda4cc91290 SystemUUID:ec2e9d55-2116-a3a5-c39b-4fda4cc91290 BootID:1b876685-fd91-487b-b238-990d0a91425e Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:88:85:3f:ed:05 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:88:85:3f:ed:05 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:c2:af:26:12:06 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 18:34:34.365660 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.365658    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 18:34:34.365745 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.365728    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 18:34:34.368624 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.368599    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 18:34:34.368771 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.368626    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-187.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 18:34:34.369316 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.369306    2579 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 18:34:34.369316 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.369317    2579 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 18:34:34.369377 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.369333    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 18:34:34.369654 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.369639    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 18:34:34.370005 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.369995    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 18:34:34.370719 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.370710    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 18:34:34.372427 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.372412    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 18:34:34.372550 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.372541    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 18:34:34.372588 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.372571    2579 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 18:34:34.372588 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.372580    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 18:34:34.372663 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.372595    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 18:34:34.372663 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.372605    2579 kubelet.go:398] "Adding apiserver pod source"
Jul 29 18:34:34.372663 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.372614    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 18:34:34.373069 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.372960    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 18:34:34.373128 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.373082    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 18:34:34.378342 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.378312    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 18:34:34.379968 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.379954    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 18:34:34.380035 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.379983    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 18:34:34.381989 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.381969    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 18:34:34.382062 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.381998    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 18:34:34.382062 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.382008    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 18:34:34.382062 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.382016    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 18:34:34.382062 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.382027    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 18:34:34.382062 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.382034    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 18:34:34.382062 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.382044    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 18:34:34.382062 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.382052    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 18:34:34.382062 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.382061    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 18:34:34.382264 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.382068    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 18:34:34.382264 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.382079    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 18:34:34.382264 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.382086    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 18:34:34.382833 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.382822    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 18:34:34.382833 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.382833    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 18:34:34.384601 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.384577    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-187.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 18:34:34.384601 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.384580    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 18:34:34.386014 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.386002    2579 server.go:1268] "Started kubelet"
Jul 29 18:34:34.386140 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.386081    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 18:34:34.386195 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.386121    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 18:34:34.386236 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.386222    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 18:34:34.386642 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.386626    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-187.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 18:34:34.386967 ip-10-0-137-187 systemd[1]: Started Kubernetes Kubelet.
Jul 29 18:34:34.387706 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.387694    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 18:34:34.388211 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.388198    2579 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 18:34:34.389832 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.389811    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 18:34:34.393307 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.393289    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 18:34:34.393307 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.393295    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 18:34:34.393462 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.393353    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found"
Jul 29 18:34:34.393604 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.392669    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-187.ec2.internal.18c6d66ef77e3c1e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-187.ec2.internal,UID:ip-10-0-137-187.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-187.ec2.internal,},FirstTimestamp:2026-07-29 18:34:34.385972254 +0000 UTC m=+0.337539320,LastTimestamp:2026-07-29 18:34:34.385972254 +0000 UTC m=+0.337539320,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-187.ec2.internal,}"
Jul 29 18:34:34.393839 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.393825    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 18:34:34.393839 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.393840    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 18:34:34.393950 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.393887    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 18:34:34.394012 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.393964    2579 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 18:34:34.394012 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.393972    2579 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 18:34:34.394122 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.394108    2579 factory.go:55] Registering systemd factory
Jul 29 18:34:34.394179 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.394126    2579 factory.go:223] Registration of the systemd container factory successfully
Jul 29 18:34:34.394319 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.394307    2579 factory.go:153] Registering CRI-O factory
Jul 29 18:34:34.394383 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.394322    2579 factory.go:223] Registration of the crio container factory successfully
Jul 29 18:34:34.394383 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.394380    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 18:34:34.394479 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.394405    2579 factory.go:103] Registering Raw factory
Jul 29 18:34:34.394479 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.394420    2579 manager.go:1196] Started watching for new ooms in manager
Jul 29 18:34:34.395052 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.395040    2579 manager.go:349] Starting recovery of all containers
Jul 29 18:34:34.397238 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.397215    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 18:34:34.397334 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.397291    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-187.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 18:34:34.401018 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.400975    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 18:34:34.406219 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.406074    2579 manager.go:354] Recovery completed
Jul 29 18:34:34.410669 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.410658    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 18:34:34.412553 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.412537    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 18:34:34.412674 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.412569    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 18:34:34.412674 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.412582    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientPID"
Jul 29 18:34:34.413086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.413074    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 18:34:34.413086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.413080    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 18:34:34.413086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.413085    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 18:34:34.413185 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.413099    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 18:34:34.413762 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.413749    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 18:34:34.413797 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.413774    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 18:34:34.413797 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.413783    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientPID"
Jul 29 18:34:34.414644 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.414584    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-187.ec2.internal.18c6d66ef913db1d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-187.ec2.internal,UID:ip-10-0-137-187.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-187.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-187.ec2.internal,},FirstTimestamp:2026-07-29 18:34:34.412555037 +0000 UTC m=+0.364122104,LastTimestamp:2026-07-29 18:34:34.412555037 +0000 UTC m=+0.364122104,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-187.ec2.internal,}"
Jul 29 18:34:34.415097 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.415087    2579 policy_none.go:49] "None policy: Start"
Jul 29 18:34:34.415137 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.415101    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 18:34:34.415137 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.415111    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 18:34:34.416046 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.416034    2579 policy_none.go:47] "Start"
Jul 29 18:34:34.417835 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.417819    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mv7pw"
Jul 29 18:34:34.422835 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.422777    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-187.ec2.internal.18c6d66ef9142d3b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-187.ec2.internal,UID:ip-10-0-137-187.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-187.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-187.ec2.internal,},FirstTimestamp:2026-07-29 18:34:34.412576059 +0000 UTC m=+0.364143125,LastTimestamp:2026-07-29 18:34:34.412576059 +0000 UTC m=+0.364143125,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-187.ec2.internal,}"
Jul 29 18:34:34.425713 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.425698    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mv7pw"
Jul 29 18:34:34.460025 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.460007    2579 manager.go:337] "Starting Device Plugin manager"
Jul 29 18:34:34.460117 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.460043    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 18:34:34.460117 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.460052    2579 server.go:87] "Starting device plugin registration server"
Jul 29 18:34:34.460298 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.460285    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 18:34:34.460344 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.460302    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 18:34:34.460415 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.460400    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 18:34:34.460520 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.460483    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 18:34:34.460520 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.460515    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 18:34:34.460883 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.460864    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 18:34:34.460946 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.460906    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-187.ec2.internal\" not found"
Jul 29 18:34:34.560809 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.560775    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 18:34:34.562743 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.562727    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 18:34:34.562830 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.562754    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 18:34:34.562830 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.562764    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientPID"
Jul 29 18:34:34.562830 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.562790    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-187.ec2.internal"
Jul 29 18:34:34.574248 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.574215    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-187.ec2.internal"
Jul 29 18:34:34.574248 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.574247    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-187.ec2.internal\": node \"ip-10-0-137-187.ec2.internal\" not found"
Jul 29 18:34:34.578702 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.578679    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 18:34:34.578834 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.578717    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 18:34:34.578834 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.578736    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 18:34:34.578834 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.578770    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 18:34:34.581714 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.581694    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 18:34:34.601441 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.601419    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found"
Jul 29 18:34:34.679359 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.679325    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal"]
Jul 29 18:34:34.679527 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.679410    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 18:34:34.682144 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.682129    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 18:34:34.682198 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.682158    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 18:34:34.682198 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.682171    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientPID"
Jul 29 18:34:34.683538 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.683523    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 18:34:34.683634 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.683618    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal"
Jul 29 18:34:34.683681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.683650    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 18:34:34.684909 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.684895    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 18:34:34.684995 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.684918    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 18:34:34.684995 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.684927    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientPID"
Jul 29 18:34:34.685247 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.685233    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 18:34:34.685304 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.685258    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 18:34:34.685304 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.685268    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientPID"
Jul 29 18:34:34.686839 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.686825    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal"
Jul 29 18:34:34.686908 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.686848    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 18:34:34.687747 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.687734    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 18:34:34.687802 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.687764    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 18:34:34.687802 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.687775    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientPID"
Jul 29 18:34:34.694882 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.694857    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/89c9dea4caba128f755081430d94df5d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal\" (UID: \"89c9dea4caba128f755081430d94df5d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal"
Jul 29 18:34:34.694985 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.694890    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89c9dea4caba128f755081430d94df5d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal\" (UID: \"89c9dea4caba128f755081430d94df5d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal"
Jul 29 18:34:34.694985 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.694913    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/edd1a1a78d8e70eee8b9b046cdfe44df-config\") pod \"kube-apiserver-proxy-ip-10-0-137-187.ec2.internal\" (UID: \"edd1a1a78d8e70eee8b9b046cdfe44df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal"
Jul 29 18:34:34.701916 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.701886    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found"
Jul 29 18:34:34.707314 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.707289    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-187.ec2.internal\" not found" node="ip-10-0-137-187.ec2.internal"
Jul 29 18:34:34.711547 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.711530    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-187.ec2.internal\" not found" node="ip-10-0-137-187.ec2.internal"
Jul 29 18:34:34.795635 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.795603    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/89c9dea4caba128f755081430d94df5d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal\" (UID: \"89c9dea4caba128f755081430d94df5d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal"
Jul 29 18:34:34.795635 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.795638    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89c9dea4caba128f755081430d94df5d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal\" (UID: \"89c9dea4caba128f755081430d94df5d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal"
Jul 29 18:34:34.795842 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.795662    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/edd1a1a78d8e70eee8b9b046cdfe44df-config\") pod \"kube-apiserver-proxy-ip-10-0-137-187.ec2.internal\" (UID: \"edd1a1a78d8e70eee8b9b046cdfe44df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal"
Jul 29 18:34:34.795842 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.795695    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/edd1a1a78d8e70eee8b9b046cdfe44df-config\") pod \"kube-apiserver-proxy-ip-10-0-137-187.ec2.internal\" (UID: \"edd1a1a78d8e70eee8b9b046cdfe44df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal"
Jul 29 18:34:34.795842 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.795701    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89c9dea4caba128f755081430d94df5d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal\" (UID: \"89c9dea4caba128f755081430d94df5d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal"
Jul 29 18:34:34.795842 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:34.795736    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/89c9dea4caba128f755081430d94df5d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal\" (UID: \"89c9dea4caba128f755081430d94df5d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal"
Jul 29 18:34:34.802739 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.802708    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found"
Jul 29 18:34:34.903421 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:34.903339    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found"
Jul 29 18:34:35.003767 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:35.003733    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found"
Jul 29 18:34:35.009871 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.009847    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal"
Jul 29 18:34:35.015005 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.014465    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal"
Jul 29 18:34:35.104807 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:35.104772    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found"
Jul 29 18:34:35.205215 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:35.205137    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found"
Jul 29 18:34:35.305570 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:35.305537    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found"
Jul 29 18:34:35.314226 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.314202    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 18:34:35.338538 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.338494    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 18:34:35.339014 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.338651    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 18:34:35.339014 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.338679    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 18:34:35.373785 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.373747    2579 apiserver.go:52] "Watching apiserver"
Jul 29 18:34:35.390578 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.390547    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 18:34:35.394071 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.393418    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 18:34:35.394071 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.393694    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-d9d6c","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5","openshift-dns/node-resolver-bd5l5","openshift-image-registry/node-ca-6vrk4","openshift-multus/multus-nnckl","openshift-network-operator/iptables-alerter-n5bwh","openshift-ovn-kubernetes/ovnkube-node-998nt","kube-system/konnectivity-agent-9f7kk","openshift-cluster-node-tuning-operator/tuned-m548l","openshift-multus/multus-additional-cni-plugins-2kjtm","openshift-multus/network-metrics-daemon-2vbxt","openshift-network-diagnostics/network-check-target-2dp6b"]
Jul 29 18:34:35.394546 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.394524    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal"
Jul 29 18:34:35.395184 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.395166    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:35.395257 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:35.395241    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:34:35.396202 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.396188    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.397557 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.397520    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bd5l5"
Jul 29 18:34:35.398347 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.398334    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 18:34:35.398587 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.398572    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret\") pod \"global-pull-secret-syncer-d9d6c\" (UID: \"32e483de-2fdb-429d-bb01-5f7ea6a0e46b\") " pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:35.398640 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.398596    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 18:34:35.398640 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.398606    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-kubelet-config\") pod \"global-pull-secret-syncer-d9d6c\" (UID: \"32e483de-2fdb-429d-bb01-5f7ea6a0e46b\") " pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:35.398640 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.398579    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 18:34:35.398640 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.398629    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-dbus\") pod \"global-pull-secret-syncer-d9d6c\" (UID: \"32e483de-2fdb-429d-bb01-5f7ea6a0e46b\") " pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:35.398640 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.398599    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mnn6z\""
Jul 29 18:34:35.398816 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.398698    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6vrk4"
Jul 29 18:34:35.399622 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.399610    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 18:34:35.399766 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.399752    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.399836 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.399818    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 18:34:35.399900 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.399885    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-47sht\""
Jul 29 18:34:35.400699 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.400683    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-n5bwh"
Jul 29 18:34:35.400991 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.400978    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zpgb4\""
Jul 29 18:34:35.401126 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.401114    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 18:34:35.401185 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.401116    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 18:34:35.401466 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.401453    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 18:34:35.401934 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.401921    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.402172 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.402154    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 18:34:35.402285 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.402267    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 18:34:35.403210 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.403188    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9f7kk"
Jul 29 18:34:35.403314 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.403259    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 18:34:35.403314 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.403308    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 18:34:35.403826 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.403800    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 18:34:35.403826 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.403800    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 18:34:35.403963 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.403807    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qj7qr\""
Jul 29 18:34:35.403963 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.403846    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-d5zlt\""
Jul 29 18:34:35.403963 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.403843    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 18:34:35.404197 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.404186    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 18:34:35.404420 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.404401    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 18:34:35.404525 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.404495    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.404849 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.404828    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 18:34:35.405686 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.405669    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 18:34:35.405686 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.405679    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 18:34:35.405832 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.405691    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kd5rp\""
Jul 29 18:34:35.405832 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.405672    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 18:34:35.405832 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.405796    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.406124 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.406048    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 18:34:35.406124 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.406120    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 18:34:35.406259 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.406150    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pdt4b\""
Jul 29 18:34:35.406680 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.406667    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 18:34:35.406746 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.406731    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 18:34:35.406913 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.406900    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 18:34:35.406968 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.406948    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:35.407012 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:35.406993    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:34:35.407165 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.407153    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9g2zt\""
Jul 29 18:34:35.407848 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.407834    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 18:34:35.408030 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.408018    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 18:34:35.408158 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.408142    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:35.408216 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:35.408202    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:34:35.408300 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.408289    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nlvld\""
Jul 29 18:34:35.410372 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.410357    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 18:34:35.410442 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.410377    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal"]
Jul 29 18:34:35.410442 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.410416    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal"
Jul 29 18:34:35.419863 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.419848    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 18:34:35.419946 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.419928    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal"]
Jul 29 18:34:35.429877 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.429830    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 18:29:34 +0000 UTC" deadline="2028-03-04 09:30:59.895522756 +0000 UTC"
Jul 29 18:34:35.429991 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.429871    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14006h56m24.465655689s"
Jul 29 18:34:35.448641 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.448604    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v86lk"
Jul 29 18:34:35.455695 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.455650    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v86lk"
Jul 29 18:34:35.495026 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.495003    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 18:34:35.498885 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.498861    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.499007 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.498896    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret\") pod \"global-pull-secret-syncer-d9d6c\" (UID: \"32e483de-2fdb-429d-bb01-5f7ea6a0e46b\") " pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:35.499007 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.498920    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-run-netns\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.499007 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.498941    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mzc4\" (UniqueName: \"kubernetes.io/projected/dcfea240-b8d3-4728-b24f-4fe8268f6d05-kube-api-access-8mzc4\") pod \"node-ca-6vrk4\" (UID: \"dcfea240-b8d3-4728-b24f-4fe8268f6d05\") " pod="openshift-image-registry/node-ca-6vrk4"
Jul 29 18:34:35.499007 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.498962    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-system-cni-dir\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.499007 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.498982    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w52mm\" (UniqueName: \"kubernetes.io/projected/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-kube-api-access-w52mm\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.499007 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:35.498994    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 18:34:35.499218 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499011    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-cni-bin\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.499218 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499069    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/98679e95-83f4-48b6-8295-f097eebbd6bb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.499218 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:35.499083    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret podName:32e483de-2fdb-429d-bb01-5f7ea6a0e46b nodeName:}" failed. No retries permitted until 2026-07-29 18:34:35.999062487 +0000 UTC m=+1.950629545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret") pod "global-pull-secret-syncer-d9d6c" (UID: "32e483de-2fdb-429d-bb01-5f7ea6a0e46b") : object "kube-system"/"original-pull-secret" not registered
Jul 29 18:34:35.499218 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499112    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-socket-dir\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.499218 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499135    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-sysctl-d\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.499218 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499153    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-run\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.499218 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499175    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhzfz\" (UniqueName: \"kubernetes.io/projected/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-kube-api-access-vhzfz\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.499218 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499193    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dcfea240-b8d3-4728-b24f-4fe8268f6d05-host\") pod \"node-ca-6vrk4\" (UID: \"dcfea240-b8d3-4728-b24f-4fe8268f6d05\") " pod="openshift-image-registry/node-ca-6vrk4"
Jul 29 18:34:35.499218 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499213    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-etc-openvswitch\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.499536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499233    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/98679e95-83f4-48b6-8295-f097eebbd6bb-os-release\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.499536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499253    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-systemd\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.499536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1aa370e-70cd-4cbc-9af4-508b7a39b9fc-tmp-dir\") pod \"node-resolver-bd5l5\" (UID: \"d1aa370e-70cd-4cbc-9af4-508b7a39b9fc\") " pod="openshift-dns/node-resolver-bd5l5"
Jul 29 18:34:35.499536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499291    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-var-lib-cni-multus\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.499536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499307    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-multus-conf-dir\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.499536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499342    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98679e95-83f4-48b6-8295-f097eebbd6bb-cnibin\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.499536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499361    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsw6v\" (UniqueName: \"kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v\") pod \"network-check-target-2dp6b\" (UID: \"cbaacddf-394f-43c1-b02b-f85d651ebe28\") " pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:35.499536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499381    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-run-multus-certs\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.499536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499415    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpxk9\" (UniqueName: \"kubernetes.io/projected/22c35b99-bdea-4dc0-bfd3-39ff0b9756c3-kube-api-access-tpxk9\") pod \"iptables-alerter-n5bwh\" (UID: \"22c35b99-bdea-4dc0-bfd3-39ff0b9756c3\") " pod="openshift-network-operator/iptables-alerter-n5bwh"
Jul 29 18:34:35.499536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499435    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98679e95-83f4-48b6-8295-f097eebbd6bb-system-cni-dir\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.499536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499450    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-dbus\") pod \"global-pull-secret-syncer-d9d6c\" (UID: \"32e483de-2fdb-429d-bb01-5f7ea6a0e46b\") " pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:35.499536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499464    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-systemd-units\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.499536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499481    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-slash\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.499536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499514    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw6jg\" (UniqueName: \"kubernetes.io/projected/ef2dbc84-9591-4b1b-8613-7887422229f2-kube-api-access-kw6jg\") pod \"network-metrics-daemon-2vbxt\" (UID: \"ef2dbc84-9591-4b1b-8613-7887422229f2\") " pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:35.499536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499531    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8277d741-f79d-4ff1-bf93-7b8443daa3be-multus-daemon-config\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.499997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499548    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.499997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499563    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-modprobe-d\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.499997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499579    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-lib-modules\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.499997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499585    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-dbus\") pod \"global-pull-secret-syncer-d9d6c\" (UID: \"32e483de-2fdb-429d-bb01-5f7ea6a0e46b\") " pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:35.499997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499623    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-node-log\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.499997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499637    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-run-ovn-kubernetes\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.499997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499658    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64tq9\" (UniqueName: \"kubernetes.io/projected/98679e95-83f4-48b6-8295-f097eebbd6bb-kube-api-access-64tq9\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.499997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499680    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8277d741-f79d-4ff1-bf93-7b8443daa3be-cni-binary-copy\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.499997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499695    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-etc-selinux\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.499997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499708    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1aa370e-70cd-4cbc-9af4-508b7a39b9fc-hosts-file\") pod \"node-resolver-bd5l5\" (UID: \"d1aa370e-70cd-4cbc-9af4-508b7a39b9fc\") " pod="openshift-dns/node-resolver-bd5l5"
Jul 29 18:34:35.499997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499721    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-multus-socket-dir-parent\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.499997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499753    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-var-lib-kubelet\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.499997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499792    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/22c35b99-bdea-4dc0-bfd3-39ff0b9756c3-iptables-alerter-script\") pod \"iptables-alerter-n5bwh\" (UID: \"22c35b99-bdea-4dc0-bfd3-39ff0b9756c3\") " pod="openshift-network-operator/iptables-alerter-n5bwh"
Jul 29 18:34:35.499997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499822    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-log-socket\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.499997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499843    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-ovnkube-config\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.499997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499870    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/89b15060-8d5e-4bc9-84ba-4a7a4b7cb9d8-konnectivity-ca\") pod \"konnectivity-agent-9f7kk\" (UID: \"89b15060-8d5e-4bc9-84ba-4a7a4b7cb9d8\") " pod="kube-system/konnectivity-agent-9f7kk"
Jul 29 18:34:35.500585 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499893    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-sysconfig\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.500585 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499907    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-sys\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.500585 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499940    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-run-systemd\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.500585 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499963    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/98679e95-83f4-48b6-8295-f097eebbd6bb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.500585 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499977    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-device-dir\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.500585 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.499990    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-sys-fs\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.500585 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500010    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9grm6\" (UniqueName: \"kubernetes.io/projected/0f98ae4a-4824-46aa-a26a-438767ceec92-kube-api-access-9grm6\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.500585 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500028    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-ovnkube-script-lib\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.500585 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500048    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-run-ovn\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.500585 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500071    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-var-lib-kubelet\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.500585 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500096    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-host\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.500585 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500117    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-cnibin\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.500585 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500144    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-os-release\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.500585 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500172    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/22c35b99-bdea-4dc0-bfd3-39ff0b9756c3-host-slash\") pod \"iptables-alerter-n5bwh\" (UID: \"22c35b99-bdea-4dc0-bfd3-39ff0b9756c3\") " pod="openshift-network-operator/iptables-alerter-n5bwh"
Jul 29 18:34:35.500585 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500195    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-tuned\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.500585 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500213    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs\") pod \"network-metrics-daemon-2vbxt\" (UID: \"ef2dbc84-9591-4b1b-8613-7887422229f2\") " pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500234    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-run-k8s-cni-cncf-io\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500252    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-etc-kubernetes\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500285    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-ovn-node-metrics-cert\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500309    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-hostroot\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500336    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-env-overrides\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500360    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-kubelet\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500381    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-run-openvswitch\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500400    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-kubernetes\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500422    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6xjb\" (UniqueName: \"kubernetes.io/projected/d1aa370e-70cd-4cbc-9af4-508b7a39b9fc-kube-api-access-z6xjb\") pod \"node-resolver-bd5l5\" (UID: \"d1aa370e-70cd-4cbc-9af4-508b7a39b9fc\") " pod="openshift-dns/node-resolver-bd5l5"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500464    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-var-lib-cni-bin\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500492    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-tmp\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500549    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-var-lib-openvswitch\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500572    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-cni-netd\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500606    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98679e95-83f4-48b6-8295-f097eebbd6bb-cni-binary-copy\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500636    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dcfea240-b8d3-4728-b24f-4fe8268f6d05-serviceca\") pod \"node-ca-6vrk4\" (UID: \"dcfea240-b8d3-4728-b24f-4fe8268f6d05\") " pod="openshift-image-registry/node-ca-6vrk4"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500657    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/89b15060-8d5e-4bc9-84ba-4a7a4b7cb9d8-agent-certs\") pod \"konnectivity-agent-9f7kk\" (UID: \"89b15060-8d5e-4bc9-84ba-4a7a4b7cb9d8\") " pod="kube-system/konnectivity-agent-9f7kk"
Jul 29 18:34:35.501112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500677    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-sysctl-conf\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.501610 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500698    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-multus-cni-dir\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.501610 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500720    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jjwz\" (UniqueName: \"kubernetes.io/projected/8277d741-f79d-4ff1-bf93-7b8443daa3be-kube-api-access-8jjwz\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.501610 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500745    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-kubelet-config\") pod \"global-pull-secret-syncer-d9d6c\" (UID: \"32e483de-2fdb-429d-bb01-5f7ea6a0e46b\") " pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:35.501610 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500770    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98679e95-83f4-48b6-8295-f097eebbd6bb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.501610 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500790    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-registration-dir\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.501610 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500805    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-run-netns\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.501610 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.500820    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-kubelet-config\") pod \"global-pull-secret-syncer-d9d6c\" (UID: \"32e483de-2fdb-429d-bb01-5f7ea6a0e46b\") " pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:35.565330 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:35.565289    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89c9dea4caba128f755081430d94df5d.slice/crio-e7e3f8c8d7d1136e08d2572cb35d9c62fa59b5ef348c161d66f314efaa66de7c WatchSource:0}: Error finding container e7e3f8c8d7d1136e08d2572cb35d9c62fa59b5ef348c161d66f314efaa66de7c: Status 404 returned error can't find the container with id e7e3f8c8d7d1136e08d2572cb35d9c62fa59b5ef348c161d66f314efaa66de7c
Jul 29 18:34:35.565539 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:35.565521    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedd1a1a78d8e70eee8b9b046cdfe44df.slice/crio-2e4894c93de3b348c2370efef3442c30e96c4bb97945f7ca70d105ad6eb18604 WatchSource:0}: Error finding container 2e4894c93de3b348c2370efef3442c30e96c4bb97945f7ca70d105ad6eb18604: Status 404 returned error can't find the container with id 2e4894c93de3b348c2370efef3442c30e96c4bb97945f7ca70d105ad6eb18604
Jul 29 18:34:35.572087 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.570779    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:34:35.582197 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.582147    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal" event={"ID":"edd1a1a78d8e70eee8b9b046cdfe44df","Type":"ContainerStarted","Data":"2e4894c93de3b348c2370efef3442c30e96c4bb97945f7ca70d105ad6eb18604"}
Jul 29 18:34:35.583254 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.583230    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" event={"ID":"89c9dea4caba128f755081430d94df5d","Type":"ContainerStarted","Data":"e7e3f8c8d7d1136e08d2572cb35d9c62fa59b5ef348c161d66f314efaa66de7c"}
Jul 29 18:34:35.588291 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.588271    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 18:34:35.601676 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601650    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-run-systemd\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.601776 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601681    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/98679e95-83f4-48b6-8295-f097eebbd6bb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.601776 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601697    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-device-dir\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.601776 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-sys-fs\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.601776 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601725    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9grm6\" (UniqueName: \"kubernetes.io/projected/0f98ae4a-4824-46aa-a26a-438767ceec92-kube-api-access-9grm6\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.601776 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601743    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-run-systemd\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.601776 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601775    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-sys-fs\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601781    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-device-dir\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-ovnkube-script-lib\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601823    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-run-ovn\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601842    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-var-lib-kubelet\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601862    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-host\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601882    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-cnibin\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601886    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-run-ovn\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601905    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-os-release\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601926    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/22c35b99-bdea-4dc0-bfd3-39ff0b9756c3-host-slash\") pod \"iptables-alerter-n5bwh\" (UID: \"22c35b99-bdea-4dc0-bfd3-39ff0b9756c3\") " pod="openshift-network-operator/iptables-alerter-n5bwh"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601932    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-var-lib-kubelet\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601936    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-cnibin\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601944    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-host\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601971    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/22c35b99-bdea-4dc0-bfd3-39ff0b9756c3-host-slash\") pod \"iptables-alerter-n5bwh\" (UID: \"22c35b99-bdea-4dc0-bfd3-39ff0b9756c3\") " pod="openshift-network-operator/iptables-alerter-n5bwh"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601945    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-tuned\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.601978    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-os-release\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602007    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs\") pod \"network-metrics-daemon-2vbxt\" (UID: \"ef2dbc84-9591-4b1b-8613-7887422229f2\") " pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602026    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-run-k8s-cni-cncf-io\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.602086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-etc-kubernetes\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602056    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-ovn-node-metrics-cert\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602076    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-hostroot\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602098    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-env-overrides\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:35.602111    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602119    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-kubelet\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602124    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-etc-kubernetes\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-run-openvswitch\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602167    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-run-openvswitch\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:35.602194    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs podName:ef2dbc84-9591-4b1b-8613-7887422229f2 nodeName:}" failed. No retries permitted until 2026-07-29 18:34:36.102174582 +0000 UTC m=+2.053741646 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs") pod "network-metrics-daemon-2vbxt" (UID: "ef2dbc84-9591-4b1b-8613-7887422229f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602202    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-hostroot\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602215    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602211    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-kubernetes\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602251    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6xjb\" (UniqueName: \"kubernetes.io/projected/d1aa370e-70cd-4cbc-9af4-508b7a39b9fc-kube-api-access-z6xjb\") pod \"node-resolver-bd5l5\" (UID: \"d1aa370e-70cd-4cbc-9af4-508b7a39b9fc\") " pod="openshift-dns/node-resolver-bd5l5"
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602274    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-var-lib-cni-bin\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602306    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-tmp\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602319    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-kubernetes\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602325    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-var-lib-openvswitch\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.602883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602095    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-run-k8s-cni-cncf-io\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602346    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-cni-netd\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602347    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/98679e95-83f4-48b6-8295-f097eebbd6bb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98679e95-83f4-48b6-8295-f097eebbd6bb-cni-binary-copy\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602370    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-kubelet\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602405    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dcfea240-b8d3-4728-b24f-4fe8268f6d05-serviceca\") pod \"node-ca-6vrk4\" (UID: \"dcfea240-b8d3-4728-b24f-4fe8268f6d05\") " pod="openshift-image-registry/node-ca-6vrk4"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602427    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/89b15060-8d5e-4bc9-84ba-4a7a4b7cb9d8-agent-certs\") pod \"konnectivity-agent-9f7kk\" (UID: \"89b15060-8d5e-4bc9-84ba-4a7a4b7cb9d8\") " pod="kube-system/konnectivity-agent-9f7kk"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602447    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-sysctl-conf\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602460    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-var-lib-openvswitch\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602470    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-multus-cni-dir\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602492    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-ovnkube-script-lib\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602526    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-cni-netd\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jjwz\" (UniqueName: \"kubernetes.io/projected/8277d741-f79d-4ff1-bf93-7b8443daa3be-kube-api-access-8jjwz\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602556    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-env-overrides\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602570    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98679e95-83f4-48b6-8295-f097eebbd6bb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602598    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-registration-dir\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602635    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-sysctl-conf\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.603681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602644    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-registration-dir\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602681    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-run-netns\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602739    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98679e95-83f4-48b6-8295-f097eebbd6bb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602752    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-run-netns\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602775    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-var-lib-cni-bin\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602809    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mzc4\" (UniqueName: \"kubernetes.io/projected/dcfea240-b8d3-4728-b24f-4fe8268f6d05-kube-api-access-8mzc4\") pod \"node-ca-6vrk4\" (UID: \"dcfea240-b8d3-4728-b24f-4fe8268f6d05\") " pod="openshift-image-registry/node-ca-6vrk4"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602833    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-system-cni-dir\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602854    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602857    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w52mm\" (UniqueName: \"kubernetes.io/projected/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-kube-api-access-w52mm\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602888    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-cni-bin\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602914    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/98679e95-83f4-48b6-8295-f097eebbd6bb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-socket-dir\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602965    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dcfea240-b8d3-4728-b24f-4fe8268f6d05-serviceca\") pod \"node-ca-6vrk4\" (UID: \"dcfea240-b8d3-4728-b24f-4fe8268f6d05\") " pod="openshift-image-registry/node-ca-6vrk4"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602964    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98679e95-83f4-48b6-8295-f097eebbd6bb-cni-binary-copy\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.602979    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-sysctl-d\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603000    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-run\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.604401 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603021    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhzfz\" (UniqueName: \"kubernetes.io/projected/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-kube-api-access-vhzfz\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603032    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-multus-cni-dir\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dcfea240-b8d3-4728-b24f-4fe8268f6d05-host\") pod \"node-ca-6vrk4\" (UID: \"dcfea240-b8d3-4728-b24f-4fe8268f6d05\") " pod="openshift-image-registry/node-ca-6vrk4"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603131    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-system-cni-dir\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603165    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-run\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603266    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-run-netns\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603299    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-run-netns\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603325    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-cni-bin\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603373    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-sysctl-d\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603385    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dcfea240-b8d3-4728-b24f-4fe8268f6d05-host\") pod \"node-ca-6vrk4\" (UID: \"dcfea240-b8d3-4728-b24f-4fe8268f6d05\") " pod="openshift-image-registry/node-ca-6vrk4"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603415    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-etc-openvswitch\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603439    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/98679e95-83f4-48b6-8295-f097eebbd6bb-os-release\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603461    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-systemd\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603482    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1aa370e-70cd-4cbc-9af4-508b7a39b9fc-tmp-dir\") pod \"node-resolver-bd5l5\" (UID: \"d1aa370e-70cd-4cbc-9af4-508b7a39b9fc\") " pod="openshift-dns/node-resolver-bd5l5"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603524    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-var-lib-cni-multus\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603540    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-etc-openvswitch\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603550    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-multus-conf-dir\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603540    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-socket-dir\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.605130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603603    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-var-lib-cni-multus\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603610    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/98679e95-83f4-48b6-8295-f097eebbd6bb-os-release\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603630    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/98679e95-83f4-48b6-8295-f097eebbd6bb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603644    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-multus-conf-dir\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603673    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98679e95-83f4-48b6-8295-f097eebbd6bb-cnibin\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603682    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-systemd\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603701    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsw6v\" (UniqueName: \"kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v\") pod \"network-check-target-2dp6b\" (UID: \"cbaacddf-394f-43c1-b02b-f85d651ebe28\") " pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-run-multus-certs\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603751    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpxk9\" (UniqueName: \"kubernetes.io/projected/22c35b99-bdea-4dc0-bfd3-39ff0b9756c3-kube-api-access-tpxk9\") pod \"iptables-alerter-n5bwh\" (UID: \"22c35b99-bdea-4dc0-bfd3-39ff0b9756c3\") " pod="openshift-network-operator/iptables-alerter-n5bwh"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603704    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98679e95-83f4-48b6-8295-f097eebbd6bb-cnibin\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603774    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98679e95-83f4-48b6-8295-f097eebbd6bb-system-cni-dir\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603795    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1aa370e-70cd-4cbc-9af4-508b7a39b9fc-tmp-dir\") pod \"node-resolver-bd5l5\" (UID: \"d1aa370e-70cd-4cbc-9af4-508b7a39b9fc\") " pod="openshift-dns/node-resolver-bd5l5"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603841    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98679e95-83f4-48b6-8295-f097eebbd6bb-system-cni-dir\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603798    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-run-multus-certs\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603862    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-systemd-units\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603798    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-systemd-units\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603920    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-slash\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.605854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603946    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw6jg\" (UniqueName: \"kubernetes.io/projected/ef2dbc84-9591-4b1b-8613-7887422229f2-kube-api-access-kw6jg\") pod \"network-metrics-daemon-2vbxt\" (UID: \"ef2dbc84-9591-4b1b-8613-7887422229f2\") " pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604105    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8277d741-f79d-4ff1-bf93-7b8443daa3be-multus-daemon-config\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.603994    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-slash\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604305    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604630    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8277d741-f79d-4ff1-bf93-7b8443daa3be-multus-daemon-config\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604672    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604707    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-modprobe-d\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604732    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-lib-modules\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604756    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-node-log\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-run-ovn-kubernetes\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604801    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64tq9\" (UniqueName: \"kubernetes.io/projected/98679e95-83f4-48b6-8295-f097eebbd6bb-kube-api-access-64tq9\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604828    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8277d741-f79d-4ff1-bf93-7b8443daa3be-cni-binary-copy\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604836    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-node-log\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604849    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-etc-selinux\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604902    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1aa370e-70cd-4cbc-9af4-508b7a39b9fc-hosts-file\") pod \"node-resolver-bd5l5\" (UID: \"d1aa370e-70cd-4cbc-9af4-508b7a39b9fc\") " pod="openshift-dns/node-resolver-bd5l5"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604919    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-lib-modules\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604924    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-multus-socket-dir-parent\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.606446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604806    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-modprobe-d\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604944    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-var-lib-kubelet\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604957    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-host-run-ovn-kubernetes\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604965    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/22c35b99-bdea-4dc0-bfd3-39ff0b9756c3-iptables-alerter-script\") pod \"iptables-alerter-n5bwh\" (UID: \"22c35b99-bdea-4dc0-bfd3-39ff0b9756c3\") " pod="openshift-network-operator/iptables-alerter-n5bwh"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.604985    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-log-socket\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605005    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-ovnkube-config\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605030    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/89b15060-8d5e-4bc9-84ba-4a7a4b7cb9d8-konnectivity-ca\") pod \"konnectivity-agent-9f7kk\" (UID: \"89b15060-8d5e-4bc9-84ba-4a7a4b7cb9d8\") " pod="kube-system/konnectivity-agent-9f7kk"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605052    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-sysconfig\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-sys\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605148    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-sys\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605170    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-host-var-lib-kubelet\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605202    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1aa370e-70cd-4cbc-9af4-508b7a39b9fc-hosts-file\") pod \"node-resolver-bd5l5\" (UID: \"d1aa370e-70cd-4cbc-9af4-508b7a39b9fc\") " pod="openshift-dns/node-resolver-bd5l5"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605226    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8277d741-f79d-4ff1-bf93-7b8443daa3be-multus-socket-dir-parent\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605300    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f98ae4a-4824-46aa-a26a-438767ceec92-etc-selinux\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605401    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-log-socket\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-sysconfig\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605623    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-etc-tuned\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605643    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-tmp\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.606921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605643    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/22c35b99-bdea-4dc0-bfd3-39ff0b9756c3-iptables-alerter-script\") pod \"iptables-alerter-n5bwh\" (UID: \"22c35b99-bdea-4dc0-bfd3-39ff0b9756c3\") " pod="openshift-network-operator/iptables-alerter-n5bwh"
Jul 29 18:34:35.607470 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605708    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-ovn-node-metrics-cert\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.607470 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605710    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8277d741-f79d-4ff1-bf93-7b8443daa3be-cni-binary-copy\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.607470 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605895    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/89b15060-8d5e-4bc9-84ba-4a7a4b7cb9d8-konnectivity-ca\") pod \"konnectivity-agent-9f7kk\" (UID: \"89b15060-8d5e-4bc9-84ba-4a7a4b7cb9d8\") " pod="kube-system/konnectivity-agent-9f7kk"
Jul 29 18:34:35.607470 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.605897    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/89b15060-8d5e-4bc9-84ba-4a7a4b7cb9d8-agent-certs\") pod \"konnectivity-agent-9f7kk\" (UID: \"89b15060-8d5e-4bc9-84ba-4a7a4b7cb9d8\") " pod="kube-system/konnectivity-agent-9f7kk"
Jul 29 18:34:35.607470 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.606372    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-ovnkube-config\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.610106 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.610089    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9grm6\" (UniqueName: \"kubernetes.io/projected/0f98ae4a-4824-46aa-a26a-438767ceec92-kube-api-access-9grm6\") pod \"aws-ebs-csi-driver-node-cbfb5\" (UID: \"0f98ae4a-4824-46aa-a26a-438767ceec92\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.610698 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:35.610686    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 18:34:35.610737 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:35.610702    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 18:34:35.610737 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:35.610710    2579 projected.go:196] Error preparing data for projected volume kube-api-access-hsw6v for pod openshift-network-diagnostics/network-check-target-2dp6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 18:34:35.610794 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:35.610751    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v podName:cbaacddf-394f-43c1-b02b-f85d651ebe28 nodeName:}" failed. No retries permitted until 2026-07-29 18:34:36.110740053 +0000 UTC m=+2.062307109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hsw6v" (UniqueName: "kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v") pod "network-check-target-2dp6b" (UID: "cbaacddf-394f-43c1-b02b-f85d651ebe28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 18:34:35.613105 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.613081    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6xjb\" (UniqueName: \"kubernetes.io/projected/d1aa370e-70cd-4cbc-9af4-508b7a39b9fc-kube-api-access-z6xjb\") pod \"node-resolver-bd5l5\" (UID: \"d1aa370e-70cd-4cbc-9af4-508b7a39b9fc\") " pod="openshift-dns/node-resolver-bd5l5"
Jul 29 18:34:35.613929 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.613906    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jjwz\" (UniqueName: \"kubernetes.io/projected/8277d741-f79d-4ff1-bf93-7b8443daa3be-kube-api-access-8jjwz\") pod \"multus-nnckl\" (UID: \"8277d741-f79d-4ff1-bf93-7b8443daa3be\") " pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.614057 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.614042    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mzc4\" (UniqueName: \"kubernetes.io/projected/dcfea240-b8d3-4728-b24f-4fe8268f6d05-kube-api-access-8mzc4\") pod \"node-ca-6vrk4\" (UID: \"dcfea240-b8d3-4728-b24f-4fe8268f6d05\") " pod="openshift-image-registry/node-ca-6vrk4"
Jul 29 18:34:35.614108 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.614056    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64tq9\" (UniqueName: \"kubernetes.io/projected/98679e95-83f4-48b6-8295-f097eebbd6bb-kube-api-access-64tq9\") pod \"multus-additional-cni-plugins-2kjtm\" (UID: \"98679e95-83f4-48b6-8295-f097eebbd6bb\") " pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.614231 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.614213    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw6jg\" (UniqueName: \"kubernetes.io/projected/ef2dbc84-9591-4b1b-8613-7887422229f2-kube-api-access-kw6jg\") pod \"network-metrics-daemon-2vbxt\" (UID: \"ef2dbc84-9591-4b1b-8613-7887422229f2\") " pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:35.614322 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.614310    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhzfz\" (UniqueName: \"kubernetes.io/projected/dee78bed-0518-46ae-a22c-daf0e3f0ffb1-kube-api-access-vhzfz\") pod \"tuned-m548l\" (UID: \"dee78bed-0518-46ae-a22c-daf0e3f0ffb1\") " pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.614671 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.614657    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w52mm\" (UniqueName: \"kubernetes.io/projected/a7729ae2-a418-4c3a-b3b5-1c8dfec9347e-kube-api-access-w52mm\") pod \"ovnkube-node-998nt\" (UID: \"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e\") " pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.615080 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.615067    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpxk9\" (UniqueName: \"kubernetes.io/projected/22c35b99-bdea-4dc0-bfd3-39ff0b9756c3-kube-api-access-tpxk9\") pod \"iptables-alerter-n5bwh\" (UID: \"22c35b99-bdea-4dc0-bfd3-39ff0b9756c3\") " pod="openshift-network-operator/iptables-alerter-n5bwh"
Jul 29 18:34:35.730368 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.730282    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5"
Jul 29 18:34:35.734528 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.734490    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bd5l5"
Jul 29 18:34:35.739332 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:35.739306    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f98ae4a_4824_46aa_a26a_438767ceec92.slice/crio-8d3bf0e09ab2ffd8dd6fd26528ad6da4ec210fbf2e5ee8652cf76a4877f215a7 WatchSource:0}: Error finding container 8d3bf0e09ab2ffd8dd6fd26528ad6da4ec210fbf2e5ee8652cf76a4877f215a7: Status 404 returned error can't find the container with id 8d3bf0e09ab2ffd8dd6fd26528ad6da4ec210fbf2e5ee8652cf76a4877f215a7
Jul 29 18:34:35.741907 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:35.741890    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1aa370e_70cd_4cbc_9af4_508b7a39b9fc.slice/crio-b09d70e1788de19d3c08dcf94f69323dcd15e6b943bd41518eced93fcca2ab20 WatchSource:0}: Error finding container b09d70e1788de19d3c08dcf94f69323dcd15e6b943bd41518eced93fcca2ab20: Status 404 returned error can't find the container with id b09d70e1788de19d3c08dcf94f69323dcd15e6b943bd41518eced93fcca2ab20
Jul 29 18:34:35.774618 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.774588    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6vrk4"
Jul 29 18:34:35.781865 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:35.781840    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcfea240_b8d3_4728_b24f_4fe8268f6d05.slice/crio-b77b63bbc443f9764a2d5b657dfd27f7704af168affe7a8e001f8dfab8606bbe WatchSource:0}: Error finding container b77b63bbc443f9764a2d5b657dfd27f7704af168affe7a8e001f8dfab8606bbe: Status 404 returned error can't find the container with id b77b63bbc443f9764a2d5b657dfd27f7704af168affe7a8e001f8dfab8606bbe
Jul 29 18:34:35.793624 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.793601    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nnckl"
Jul 29 18:34:35.800280 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:35.800256    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8277d741_f79d_4ff1_bf93_7b8443daa3be.slice/crio-a0ccb1fbf4a7e671b19691e3b38362c2e59b8dbc90b91fdc2811c100501e388b WatchSource:0}: Error finding container a0ccb1fbf4a7e671b19691e3b38362c2e59b8dbc90b91fdc2811c100501e388b: Status 404 returned error can't find the container with id a0ccb1fbf4a7e671b19691e3b38362c2e59b8dbc90b91fdc2811c100501e388b
Jul 29 18:34:35.808429 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.808404    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-n5bwh"
Jul 29 18:34:35.815554 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:35.815527    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22c35b99_bdea_4dc0_bfd3_39ff0b9756c3.slice/crio-ae1167c98b3aed0dce34ab852a0cfb346d1975689c7119422fdbec21227f0484 WatchSource:0}: Error finding container ae1167c98b3aed0dce34ab852a0cfb346d1975689c7119422fdbec21227f0484: Status 404 returned error can't find the container with id ae1167c98b3aed0dce34ab852a0cfb346d1975689c7119422fdbec21227f0484
Jul 29 18:34:35.824876 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.824854    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:34:35.830366 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.830345    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9f7kk"
Jul 29 18:34:35.832297 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:35.832241    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7729ae2_a418_4c3a_b3b5_1c8dfec9347e.slice/crio-4e1978b765f21fec0affdae6cc2fcc8d2afae6a053cf408ede3d7b36ec360020 WatchSource:0}: Error finding container 4e1978b765f21fec0affdae6cc2fcc8d2afae6a053cf408ede3d7b36ec360020: Status 404 returned error can't find the container with id 4e1978b765f21fec0affdae6cc2fcc8d2afae6a053cf408ede3d7b36ec360020
Jul 29 18:34:35.837414 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:35.837390    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89b15060_8d5e_4bc9_84ba_4a7a4b7cb9d8.slice/crio-b340ef424d4bc2eed9f97446ec5d3b8df419f46aec5e6ab83e742bb185ff7154 WatchSource:0}: Error finding container b340ef424d4bc2eed9f97446ec5d3b8df419f46aec5e6ab83e742bb185ff7154: Status 404 returned error can't find the container with id b340ef424d4bc2eed9f97446ec5d3b8df419f46aec5e6ab83e742bb185ff7154
Jul 29 18:34:35.838246 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.838229    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m548l"
Jul 29 18:34:35.842548 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.842527    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2kjtm"
Jul 29 18:34:35.846015 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:35.845993    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddee78bed_0518_46ae_a22c_daf0e3f0ffb1.slice/crio-6e908072fca93f40f993ce5df2c5f29353b2c7ffbf2ff9f3fe722c39ffa020b6 WatchSource:0}: Error finding container 6e908072fca93f40f993ce5df2c5f29353b2c7ffbf2ff9f3fe722c39ffa020b6: Status 404 returned error can't find the container with id 6e908072fca93f40f993ce5df2c5f29353b2c7ffbf2ff9f3fe722c39ffa020b6
Jul 29 18:34:35.849962 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:34:35.849942    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98679e95_83f4_48b6_8295_f097eebbd6bb.slice/crio-a00b813871ea1289d178fcd109d740560df193cb001bb846dc229ed4d51cadbd WatchSource:0}: Error finding container a00b813871ea1289d178fcd109d740560df193cb001bb846dc229ed4d51cadbd: Status 404 returned error can't find the container with id a00b813871ea1289d178fcd109d740560df193cb001bb846dc229ed4d51cadbd
Jul 29 18:34:35.937626 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:35.937600    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 18:34:36.008381 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:36.008224    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret\") pod \"global-pull-secret-syncer-d9d6c\" (UID: \"32e483de-2fdb-429d-bb01-5f7ea6a0e46b\") " pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:36.008381 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:36.008358    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 18:34:36.008596 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:36.008425    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret podName:32e483de-2fdb-429d-bb01-5f7ea6a0e46b nodeName:}" failed. No retries permitted until 2026-07-29 18:34:37.008406092 +0000 UTC m=+2.959973148 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret") pod "global-pull-secret-syncer-d9d6c" (UID: "32e483de-2fdb-429d-bb01-5f7ea6a0e46b") : object "kube-system"/"original-pull-secret" not registered
Jul 29 18:34:36.109375 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:36.109346    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs\") pod \"network-metrics-daemon-2vbxt\" (UID: \"ef2dbc84-9591-4b1b-8613-7887422229f2\") " pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:36.109551 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:36.109516    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 18:34:36.109620 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:36.109591    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs podName:ef2dbc84-9591-4b1b-8613-7887422229f2 nodeName:}" failed. No retries permitted until 2026-07-29 18:34:37.109565735 +0000 UTC m=+3.061132807 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs") pod "network-metrics-daemon-2vbxt" (UID: "ef2dbc84-9591-4b1b-8613-7887422229f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 18:34:36.211364 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:36.210659    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsw6v\" (UniqueName: \"kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v\") pod \"network-check-target-2dp6b\" (UID: \"cbaacddf-394f-43c1-b02b-f85d651ebe28\") " pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:36.211364 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:36.210853    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 18:34:36.211364 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:36.210873    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 18:34:36.211364 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:36.210885    2579 projected.go:196] Error preparing data for projected volume kube-api-access-hsw6v for pod openshift-network-diagnostics/network-check-target-2dp6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 18:34:36.211364 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:36.210941    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v podName:cbaacddf-394f-43c1-b02b-f85d651ebe28 nodeName:}" failed. No retries permitted until 2026-07-29 18:34:37.210925706 +0000 UTC m=+3.162492762 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hsw6v" (UniqueName: "kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v") pod "network-check-target-2dp6b" (UID: "cbaacddf-394f-43c1-b02b-f85d651ebe28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 18:34:36.214695 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:36.214494    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 18:34:36.457828 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:36.457753    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 18:29:35 +0000 UTC" deadline="2028-03-24 23:27:22.125162784 +0000 UTC"
Jul 29 18:34:36.457828 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:36.457792    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14500h52m45.667375424s"
Jul 29 18:34:36.599562 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:36.599459    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nnckl" event={"ID":"8277d741-f79d-4ff1-bf93-7b8443daa3be","Type":"ContainerStarted","Data":"a0ccb1fbf4a7e671b19691e3b38362c2e59b8dbc90b91fdc2811c100501e388b"}
Jul 29 18:34:36.604425 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:36.604345    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bd5l5" event={"ID":"d1aa370e-70cd-4cbc-9af4-508b7a39b9fc","Type":"ContainerStarted","Data":"b09d70e1788de19d3c08dcf94f69323dcd15e6b943bd41518eced93fcca2ab20"}
Jul 29 18:34:36.628356 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:36.627348    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5" event={"ID":"0f98ae4a-4824-46aa-a26a-438767ceec92","Type":"ContainerStarted","Data":"8d3bf0e09ab2ffd8dd6fd26528ad6da4ec210fbf2e5ee8652cf76a4877f215a7"}
Jul 29 18:34:36.640278 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:36.640201    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m548l" event={"ID":"dee78bed-0518-46ae-a22c-daf0e3f0ffb1","Type":"ContainerStarted","Data":"6e908072fca93f40f993ce5df2c5f29353b2c7ffbf2ff9f3fe722c39ffa020b6"}
Jul 29 18:34:36.651754 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:36.651702    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9f7kk" event={"ID":"89b15060-8d5e-4bc9-84ba-4a7a4b7cb9d8","Type":"ContainerStarted","Data":"b340ef424d4bc2eed9f97446ec5d3b8df419f46aec5e6ab83e742bb185ff7154"}
Jul 29 18:34:36.655528 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:36.655471    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-998nt" event={"ID":"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e","Type":"ContainerStarted","Data":"4e1978b765f21fec0affdae6cc2fcc8d2afae6a053cf408ede3d7b36ec360020"}
Jul 29 18:34:36.666407 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:36.666369    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-n5bwh" event={"ID":"22c35b99-bdea-4dc0-bfd3-39ff0b9756c3","Type":"ContainerStarted","Data":"ae1167c98b3aed0dce34ab852a0cfb346d1975689c7119422fdbec21227f0484"}
Jul 29 18:34:36.672692 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:36.672652    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6vrk4" event={"ID":"dcfea240-b8d3-4728-b24f-4fe8268f6d05","Type":"ContainerStarted","Data":"b77b63bbc443f9764a2d5b657dfd27f7704af168affe7a8e001f8dfab8606bbe"}
Jul 29 18:34:36.683635 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:36.683599    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2kjtm" event={"ID":"98679e95-83f4-48b6-8295-f097eebbd6bb","Type":"ContainerStarted","Data":"a00b813871ea1289d178fcd109d740560df193cb001bb846dc229ed4d51cadbd"}
Jul 29 18:34:37.016240 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:37.016200    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret\") pod \"global-pull-secret-syncer-d9d6c\" (UID: \"32e483de-2fdb-429d-bb01-5f7ea6a0e46b\") " pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:37.016435 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:37.016350    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 18:34:37.016435 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:37.016408    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret podName:32e483de-2fdb-429d-bb01-5f7ea6a0e46b nodeName:}" failed. No retries permitted until 2026-07-29 18:34:39.016390724 +0000 UTC m=+4.967957781 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret") pod "global-pull-secret-syncer-d9d6c" (UID: "32e483de-2fdb-429d-bb01-5f7ea6a0e46b") : object "kube-system"/"original-pull-secret" not registered
Jul 29 18:34:37.117627 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:37.117572    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs\") pod \"network-metrics-daemon-2vbxt\" (UID: \"ef2dbc84-9591-4b1b-8613-7887422229f2\") " pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:37.117793 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:37.117709    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 18:34:37.117793 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:37.117769    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs podName:ef2dbc84-9591-4b1b-8613-7887422229f2 nodeName:}" failed. No retries permitted until 2026-07-29 18:34:39.117752029 +0000 UTC m=+5.069319085 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs") pod "network-metrics-daemon-2vbxt" (UID: "ef2dbc84-9591-4b1b-8613-7887422229f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 18:34:37.218396 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:37.218357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsw6v\" (UniqueName: \"kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v\") pod \"network-check-target-2dp6b\" (UID: \"cbaacddf-394f-43c1-b02b-f85d651ebe28\") " pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:37.218600 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:37.218545    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 18:34:37.218600 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:37.218568    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 18:34:37.218600 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:37.218580    2579 projected.go:196] Error preparing data for projected volume kube-api-access-hsw6v for pod openshift-network-diagnostics/network-check-target-2dp6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 18:34:37.218750 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:37.218635    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v podName:cbaacddf-394f-43c1-b02b-f85d651ebe28 nodeName:}" failed. No retries permitted until 2026-07-29 18:34:39.218617331 +0000 UTC m=+5.170184387 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hsw6v" (UniqueName: "kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v") pod "network-check-target-2dp6b" (UID: "cbaacddf-394f-43c1-b02b-f85d651ebe28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 18:34:37.458212 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:37.458079    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 18:29:35 +0000 UTC" deadline="2027-12-25 15:32:42.525947064 +0000 UTC"
Jul 29 18:34:37.458212 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:37.458159    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12332h58m5.067801281s"
Jul 29 18:34:37.579587 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:37.579543    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:37.579775 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:37.579676    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:34:37.579775 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:37.579763    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:37.579890 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:37.579840    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:34:37.579950 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:37.579919    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:37.580092 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:37.580067    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:34:38.405042 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:38.404966    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 18:34:39.036351 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:39.035783    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret\") pod \"global-pull-secret-syncer-d9d6c\" (UID: \"32e483de-2fdb-429d-bb01-5f7ea6a0e46b\") " pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:39.036351 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:39.035954    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 18:34:39.036351 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:39.036016    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret podName:32e483de-2fdb-429d-bb01-5f7ea6a0e46b nodeName:}" failed. No retries permitted until 2026-07-29 18:34:43.035997321 +0000 UTC m=+8.987564380 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret") pod "global-pull-secret-syncer-d9d6c" (UID: "32e483de-2fdb-429d-bb01-5f7ea6a0e46b") : object "kube-system"/"original-pull-secret" not registered
Jul 29 18:34:39.136399 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:39.136351    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs\") pod \"network-metrics-daemon-2vbxt\" (UID: \"ef2dbc84-9591-4b1b-8613-7887422229f2\") " pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:39.137060 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:39.136611    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 18:34:39.137060 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:39.136683    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs podName:ef2dbc84-9591-4b1b-8613-7887422229f2 nodeName:}" failed. No retries permitted until 2026-07-29 18:34:43.136661892 +0000 UTC m=+9.088228962 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs") pod "network-metrics-daemon-2vbxt" (UID: "ef2dbc84-9591-4b1b-8613-7887422229f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 18:34:39.237795 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:39.237665    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsw6v\" (UniqueName: \"kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v\") pod \"network-check-target-2dp6b\" (UID: \"cbaacddf-394f-43c1-b02b-f85d651ebe28\") " pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:39.237999 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:39.237903    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 18:34:39.237999 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:39.237925    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 18:34:39.237999 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:39.237937    2579 projected.go:196] Error preparing data for projected volume kube-api-access-hsw6v for pod openshift-network-diagnostics/network-check-target-2dp6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 18:34:39.237999 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:39.237994    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v podName:cbaacddf-394f-43c1-b02b-f85d651ebe28 nodeName:}" failed. No retries permitted until 2026-07-29 18:34:43.23797757 +0000 UTC m=+9.189544629 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hsw6v" (UniqueName: "kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v") pod "network-check-target-2dp6b" (UID: "cbaacddf-394f-43c1-b02b-f85d651ebe28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 18:34:39.579987 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:39.579952    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:39.580187 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:39.580106    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:34:39.580641 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:39.580619    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:39.580759 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:39.580737    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:34:39.580862 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:39.580848    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:39.580964 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:39.580947    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:34:41.579610 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:41.579569    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:41.579610 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:41.579598    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:41.580157 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:41.579616    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:41.580157 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:41.579730    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:34:41.580157 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:41.579808    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:34:41.580157 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:41.579838    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:34:43.071283 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:43.071167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret\") pod \"global-pull-secret-syncer-d9d6c\" (UID: \"32e483de-2fdb-429d-bb01-5f7ea6a0e46b\") " pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:43.071754 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:43.071335    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 18:34:43.071754 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:43.071414    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret podName:32e483de-2fdb-429d-bb01-5f7ea6a0e46b nodeName:}" failed. No retries permitted until 2026-07-29 18:34:51.071392115 +0000 UTC m=+17.022959183 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret") pod "global-pull-secret-syncer-d9d6c" (UID: "32e483de-2fdb-429d-bb01-5f7ea6a0e46b") : object "kube-system"/"original-pull-secret" not registered
Jul 29 18:34:43.171777 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:43.171728    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs\") pod \"network-metrics-daemon-2vbxt\" (UID: \"ef2dbc84-9591-4b1b-8613-7887422229f2\") " pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:43.171965 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:43.171920    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 18:34:43.172030 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:43.171986    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs podName:ef2dbc84-9591-4b1b-8613-7887422229f2 nodeName:}" failed. No retries permitted until 2026-07-29 18:34:51.171967569 +0000 UTC m=+17.123534627 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs") pod "network-metrics-daemon-2vbxt" (UID: "ef2dbc84-9591-4b1b-8613-7887422229f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 18:34:43.272372 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:43.272317    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsw6v\" (UniqueName: \"kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v\") pod \"network-check-target-2dp6b\" (UID: \"cbaacddf-394f-43c1-b02b-f85d651ebe28\") " pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:43.272589 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:43.272534    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 18:34:43.272589 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:43.272559    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 18:34:43.272589 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:43.272572    2579 projected.go:196] Error preparing data for projected volume kube-api-access-hsw6v for pod openshift-network-diagnostics/network-check-target-2dp6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 18:34:43.272779 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:43.272635    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v podName:cbaacddf-394f-43c1-b02b-f85d651ebe28 nodeName:}" failed. No retries permitted until 2026-07-29 18:34:51.272614261 +0000 UTC m=+17.224181319 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hsw6v" (UniqueName: "kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v") pod "network-check-target-2dp6b" (UID: "cbaacddf-394f-43c1-b02b-f85d651ebe28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 18:34:43.580219 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:43.579962    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:43.580410 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:43.580325    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:34:43.580667 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:43.580640    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:43.580748 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:43.580648    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:43.580813 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:43.580751    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:34:43.580906 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:43.580814    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:34:45.579034 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:45.578991    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:45.579468 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:45.578991    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:45.579468 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:45.579140    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:34:45.579468 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:45.579227    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:34:45.579468 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:45.578991    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:45.579468 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:45.579319    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:34:47.579124 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:47.579087    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:47.579597 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:47.579087    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:47.579597 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:47.579104    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:47.579597 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:47.579198    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:34:47.579597 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:47.579348    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:34:47.579597 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:47.579434    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:34:49.579433 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:49.579395    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:49.579897 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:49.579438    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:49.579897 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:49.579438    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:49.579897 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:49.579569    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:34:49.579897 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:49.579659    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:34:49.579897 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:49.579728    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:34:51.129752 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:51.129708    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret\") pod \"global-pull-secret-syncer-d9d6c\" (UID: \"32e483de-2fdb-429d-bb01-5f7ea6a0e46b\") " pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:51.130266 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:51.129871    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 18:34:51.130266 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:51.129952    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret podName:32e483de-2fdb-429d-bb01-5f7ea6a0e46b nodeName:}" failed. No retries permitted until 2026-07-29 18:35:07.129930469 +0000 UTC m=+33.081497538 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret") pod "global-pull-secret-syncer-d9d6c" (UID: "32e483de-2fdb-429d-bb01-5f7ea6a0e46b") : object "kube-system"/"original-pull-secret" not registered
Jul 29 18:34:51.230786 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:51.230749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs\") pod \"network-metrics-daemon-2vbxt\" (UID: \"ef2dbc84-9591-4b1b-8613-7887422229f2\") " pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:51.230966 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:51.230868    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 18:34:51.230966 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:51.230928    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs podName:ef2dbc84-9591-4b1b-8613-7887422229f2 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:07.230912674 +0000 UTC m=+33.182479728 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs") pod "network-metrics-daemon-2vbxt" (UID: "ef2dbc84-9591-4b1b-8613-7887422229f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 18:34:51.331539 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:51.331488    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsw6v\" (UniqueName: \"kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v\") pod \"network-check-target-2dp6b\" (UID: \"cbaacddf-394f-43c1-b02b-f85d651ebe28\") " pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:51.331715 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:51.331640    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 18:34:51.331715 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:51.331657    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 18:34:51.331715 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:51.331665    2579 projected.go:196] Error preparing data for projected volume kube-api-access-hsw6v for pod openshift-network-diagnostics/network-check-target-2dp6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 18:34:51.331833 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:51.331729    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v podName:cbaacddf-394f-43c1-b02b-f85d651ebe28 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:07.331709215 +0000 UTC m=+33.283276271 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hsw6v" (UniqueName: "kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v") pod "network-check-target-2dp6b" (UID: "cbaacddf-394f-43c1-b02b-f85d651ebe28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 18:34:51.579461 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:51.579364    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:51.579461 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:51.579400    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:51.579461 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:51.579417    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:51.579748 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:51.579491    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:34:51.579748 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:51.579624    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:34:51.579748 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:51.579720    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:34:53.579667 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:53.579630    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:53.579667 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:53.579651    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:53.580197 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:53.579630    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:53.580197 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:53.579768    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:34:53.580197 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:53.579845    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:34:53.580197 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:53.579943    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:34:55.580054 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.579869    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:55.580819 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.579946    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:55.580819 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:55.580147    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:34:55.580819 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.579955    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:55.580819 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:55.580224    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:34:55.580819 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:55.580268    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:34:55.727429 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.727394    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-998nt" event={"ID":"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e","Type":"ContainerStarted","Data":"2b8a1e5a6c1c6e81925cf370a49ecde82218520a4baca569a5066cc28881856b"}
Jul 29 18:34:55.727569 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.727495    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-998nt" event={"ID":"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e","Type":"ContainerStarted","Data":"80cb74f60681b7ce779e6c37549be72ff035fed5b743b1296863fe9a31cfac63"}
Jul 29 18:34:55.727646 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.727564    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-998nt" event={"ID":"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e","Type":"ContainerStarted","Data":"a40baed660415d02743b166730562161fa7629f95a89b7bb6ca070fe07af10b7"}
Jul 29 18:34:55.727646 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.727615    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-998nt" event={"ID":"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e","Type":"ContainerStarted","Data":"87df534df37e8938865473a36aab0ad706aff7d081929a6e03faa781424d21a8"}
Jul 29 18:34:55.727732 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.727663    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-998nt" event={"ID":"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e","Type":"ContainerStarted","Data":"dbcd32d7e476715a50c1385283ed9ecf8cade5e272ab22dacda9f5216b1a56c1"}
Jul 29 18:34:55.728870 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.728846    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6vrk4" event={"ID":"dcfea240-b8d3-4728-b24f-4fe8268f6d05","Type":"ContainerStarted","Data":"657433d57ca5e3f30cf76714659550b6f90228b5d5138cf1611065c98aeea5a2"}
Jul 29 18:34:55.730378 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.730352    2579 generic.go:356] "Generic (PLEG): container finished" podID="98679e95-83f4-48b6-8295-f097eebbd6bb" containerID="b288aff79199af5270741510d7cb3d02f92a38cb05d1ec568f441c68c26e0a1c" exitCode=0
Jul 29 18:34:55.730457 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.730436    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2kjtm" event={"ID":"98679e95-83f4-48b6-8295-f097eebbd6bb","Type":"ContainerDied","Data":"b288aff79199af5270741510d7cb3d02f92a38cb05d1ec568f441c68c26e0a1c"}
Jul 29 18:34:55.731981 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.731950    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nnckl" event={"ID":"8277d741-f79d-4ff1-bf93-7b8443daa3be","Type":"ContainerStarted","Data":"32abefbca7ef4b269b0d7faaac77fb15553ecd8cd0a07a3fa5ac1c285583b6f2"}
Jul 29 18:34:55.733379 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.733357    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bd5l5" event={"ID":"d1aa370e-70cd-4cbc-9af4-508b7a39b9fc","Type":"ContainerStarted","Data":"3c756ca66c0758b2acfe986f1ad465b08eef5251c910ca79a156efd38c2b5471"}
Jul 29 18:34:55.734825 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.734763    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5" event={"ID":"0f98ae4a-4824-46aa-a26a-438767ceec92","Type":"ContainerStarted","Data":"4d8907452d1f0afabace3e2e6c5a31bdad26a05ee6017052ccc7df2c0141af9c"}
Jul 29 18:34:55.736097 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.736074    2579 generic.go:356] "Generic (PLEG): container finished" podID="89c9dea4caba128f755081430d94df5d" containerID="0aa40d72e3972f2abfd244e388c4673fa4aabc2a9b567d02a952e601d046c7e5" exitCode=0
Jul 29 18:34:55.736198 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.736153    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" event={"ID":"89c9dea4caba128f755081430d94df5d","Type":"ContainerDied","Data":"0aa40d72e3972f2abfd244e388c4673fa4aabc2a9b567d02a952e601d046c7e5"}
Jul 29 18:34:55.737457 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.737432    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal" event={"ID":"edd1a1a78d8e70eee8b9b046cdfe44df","Type":"ContainerStarted","Data":"8109c94b5418979201bc4f6dfae55cd26e201da7427687d255a90d959a082713"}
Jul 29 18:34:55.738778 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.738755    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m548l" event={"ID":"dee78bed-0518-46ae-a22c-daf0e3f0ffb1","Type":"ContainerStarted","Data":"7d3f74dbdd3735aad64b5dec9728c2892af7a4563f9170d75196154aa219419c"}
Jul 29 18:34:55.740098 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.740080    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9f7kk" event={"ID":"89b15060-8d5e-4bc9-84ba-4a7a4b7cb9d8","Type":"ContainerStarted","Data":"1100fd83e7497c99a151d9bc2f498af57b9ec2a70fb39fec70941d9947cf500c"}
Jul 29 18:34:55.742600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.742563    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6vrk4" podStartSLOduration=2.545364243 podStartE2EDuration="21.742551822s" totalImagesPullingTime="19.197187579s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 18:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:34:55.742443137 +0000 UTC m=+21.694010215" watchObservedRunningTime="2026-07-29 18:34:55.742551822 +0000 UTC m=+21.694118890"
Jul 29 18:34:55.757825 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.757776    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal" podStartSLOduration=20.757760713 podStartE2EDuration="20.757760713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 18:34:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:34:55.757230851 +0000 UTC m=+21.708797910" watchObservedRunningTime="2026-07-29 18:34:55.757760713 +0000 UTC m=+21.709327790"
Jul 29 18:34:55.773868 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.773818    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bd5l5" podStartSLOduration=2.588853765 podStartE2EDuration="21.773802619s" totalImagesPullingTime="19.184948854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 18:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:34:55.773341181 +0000 UTC m=+21.724908259" watchObservedRunningTime="2026-07-29 18:34:55.773802619 +0000 UTC m=+21.725369696"
Jul 29 18:34:55.791781 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.791730    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nnckl" podStartSLOduration=2.597425416 podStartE2EDuration="21.791714988s" totalImagesPullingTime="19.194289572s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 18:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:34:55.791561581 +0000 UTC m=+21.743128670" watchObservedRunningTime="2026-07-29 18:34:55.791714988 +0000 UTC m=+21.743282064"
Jul 29 18:34:55.806400 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.806356    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9f7kk" podStartSLOduration=2.716402398 podStartE2EDuration="21.806342627s" totalImagesPullingTime="19.089940229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:34:55.805860551 +0000 UTC m=+21.757427629" watchObservedRunningTime="2026-07-29 18:34:55.806342627 +0000 UTC m=+21.757909703"
Jul 29 18:34:55.825071 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:55.825016    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m548l" podStartSLOduration=2.69317537 podStartE2EDuration="21.825006673s" totalImagesPullingTime="19.131831303s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 18:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:34:55.824574026 +0000 UTC m=+21.776141105" watchObservedRunningTime="2026-07-29 18:34:55.825006673 +0000 UTC m=+21.776573750"
Jul 29 18:34:56.745370 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:56.745112    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" event={"ID":"89c9dea4caba128f755081430d94df5d","Type":"ContainerStarted","Data":"40ebaa3c73277926e25a72afb43a1d45804c52cb09f0bd2fa780eff57691aadd"}
Jul 29 18:34:56.748970 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:56.748941    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-998nt" event={"ID":"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e","Type":"ContainerStarted","Data":"1e80acb06fe2ad65dc800b342b42119ecb5c853cc394284437bcd15c1e8e6599"}
Jul 29 18:34:56.750643 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:56.750586    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-n5bwh" event={"ID":"22c35b99-bdea-4dc0-bfd3-39ff0b9756c3","Type":"ContainerStarted","Data":"653f2ee493cde296f002f6ba12b64f609d0ba9351fc29c60ba3802555bcc261c"}
Jul 29 18:34:56.760707 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:56.760648    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" podStartSLOduration=21.760628303 podStartE2EDuration="21.760628303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 18:34:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:34:56.759821852 +0000 UTC m=+22.711388933" watchObservedRunningTime="2026-07-29 18:34:56.760628303 +0000 UTC m=+22.712195381"
Jul 29 18:34:56.773631 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:56.773582    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-n5bwh" podStartSLOduration=3.6108310919999997 podStartE2EDuration="22.773561191s" totalImagesPullingTime="19.162730099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 18:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:34:56.773308285 +0000 UTC m=+22.724875363" watchObservedRunningTime="2026-07-29 18:34:56.773561191 +0000 UTC m=+22.725128268"
Jul 29 18:34:56.826945 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:56.826920    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 18:34:57.475994 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:57.475877    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T18:34:56.826938137Z","UUID":"bbd3ce32-eefa-44aa-a719-8ccdcd889e93","Handler":null,"Name":"","Endpoint":""}
Jul 29 18:34:57.477601 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:57.477560    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 18:34:57.477601 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:57.477607    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 18:34:57.579959 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:57.579916    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:57.579959 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:57.579946    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:57.580207 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:57.579946    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:57.580207 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:57.580046    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:34:57.580207 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:57.580155    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:34:57.580356 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:57.580236    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:34:57.755776 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:57.755688    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5" event={"ID":"0f98ae4a-4824-46aa-a26a-438767ceec92","Type":"ContainerStarted","Data":"e6676b03fde67ad788e6f8448bede04d9f2f06dc9fb027a59e17d551ba45070c"}
Jul 29 18:34:58.761608 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:58.761561    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-998nt" event={"ID":"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e","Type":"ContainerStarted","Data":"f7499586ce60999cb3c8851fb7e8557f4e297d6a545a7a9d220740d03e690817"}
Jul 29 18:34:58.764124 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:58.764088    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5" event={"ID":"0f98ae4a-4824-46aa-a26a-438767ceec92","Type":"ContainerStarted","Data":"5fe94ebfa5f8a1eea0fecb2e1bdea3a8f48b65c56528af184ff2fa6e3f28ffd6"}
Jul 29 18:34:58.783949 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:58.783891    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbfb5" podStartSLOduration=3.190201517 podStartE2EDuration="24.78387623s" totalImagesPullingTime="21.593674713s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 18:34:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:34:58.783292681 +0000 UTC m=+24.734859759" watchObservedRunningTime="2026-07-29 18:34:58.78387623 +0000 UTC m=+24.735443307"
Jul 29 18:34:59.579161 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:59.579123    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:34:59.579381 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:59.579139    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:34:59.579381 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:59.579249    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:34:59.579381 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:34:59.579262    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:34:59.579514 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:59.579357    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:34:59.579559 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:34:59.579492    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:35:00.015349 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:00.015328    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9f7kk"
Jul 29 18:35:00.015994 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:00.015971    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9f7kk"
Jul 29 18:35:00.772082 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:00.771863    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-998nt" event={"ID":"a7729ae2-a418-4c3a-b3b5-1c8dfec9347e","Type":"ContainerStarted","Data":"8f056956f1efba175dbb13e25653217868e8ca5655c718574707510180628a47"}
Jul 29 18:35:00.772256 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:00.772140    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:35:00.773731 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:00.773706    2579 generic.go:356] "Generic (PLEG): container finished" podID="98679e95-83f4-48b6-8295-f097eebbd6bb" containerID="974e315a2028577dfe1628fdb3364bb19b4f1288347238b7492e75bd14099e6c" exitCode=0
Jul 29 18:35:00.773857 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:00.773786    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2kjtm" event={"ID":"98679e95-83f4-48b6-8295-f097eebbd6bb","Type":"ContainerDied","Data":"974e315a2028577dfe1628fdb3364bb19b4f1288347238b7492e75bd14099e6c"}
Jul 29 18:35:00.773905 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:00.773878    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9f7kk"
Jul 29 18:35:00.774408 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:00.774391    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9f7kk"
Jul 29 18:35:00.788606 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:00.788578    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:35:00.799456 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:00.799398    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-998nt" podStartSLOduration=7.433276538 podStartE2EDuration="26.799381872s" totalImagesPullingTime="19.366105334s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 18:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:00.797940893 +0000 UTC m=+26.749507972" watchObservedRunningTime="2026-07-29 18:35:00.799381872 +0000 UTC m=+26.750948948"
Jul 29 18:35:01.579288 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:01.579242    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:35:01.580438 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:01.579247    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:35:01.580438 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:01.579353    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:35:01.580438 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:01.579247    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:35:01.580438 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:01.579432    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:35:01.580438 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:01.579575    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:35:01.776716 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:01.776687    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:35:01.776891 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:01.776757    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:35:01.792113 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:01.792084    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:35:02.323743 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:02.323452    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2vbxt"]
Jul 29 18:35:02.323885 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:02.323764    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:35:02.323932 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:02.323879    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:35:02.327475 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:02.327442    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2dp6b"]
Jul 29 18:35:02.327628 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:02.327560    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:35:02.327698 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:02.327644    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:35:02.329067 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:02.329042    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d9d6c"]
Jul 29 18:35:02.329160 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:02.329126    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:35:02.329212 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:02.329199    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:35:02.780534 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:02.780429    2579 generic.go:356] "Generic (PLEG): container finished" podID="98679e95-83f4-48b6-8295-f097eebbd6bb" containerID="8154e313bb0823cea7aa11c4c413edd3ed0f204df5d166feae9122664980e688" exitCode=0
Jul 29 18:35:02.780960 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:02.780534    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2kjtm" event={"ID":"98679e95-83f4-48b6-8295-f097eebbd6bb","Type":"ContainerDied","Data":"8154e313bb0823cea7aa11c4c413edd3ed0f204df5d166feae9122664980e688"}
Jul 29 18:35:03.579459 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:03.579421    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:35:03.579624 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:03.579558    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:35:04.582387 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:04.582306    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:35:04.582387 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:04.582322    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:35:04.582762 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:04.582395    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:35:04.582762 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:04.582470    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:35:04.788147 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:04.788113    2579 generic.go:356] "Generic (PLEG): container finished" podID="98679e95-83f4-48b6-8295-f097eebbd6bb" containerID="f023be4fbcc5a1f3b16dde7656df64cbd61b1a99cc900b86e793960fa3a865b0" exitCode=0
Jul 29 18:35:04.788297 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:04.788164    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2kjtm" event={"ID":"98679e95-83f4-48b6-8295-f097eebbd6bb","Type":"ContainerDied","Data":"f023be4fbcc5a1f3b16dde7656df64cbd61b1a99cc900b86e793960fa3a865b0"}
Jul 29 18:35:05.579176 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:05.579141    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:35:05.579370 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:05.579266    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:35:06.579112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:06.579081    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:35:06.579652 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:06.579087    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:35:06.579652 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:06.579216    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:35:06.579652 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:06.579261    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:35:07.141076 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:07.141041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret\") pod \"global-pull-secret-syncer-d9d6c\" (UID: \"32e483de-2fdb-429d-bb01-5f7ea6a0e46b\") " pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:35:07.141279 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:07.141204    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 18:35:07.141344 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:07.141292    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret podName:32e483de-2fdb-429d-bb01-5f7ea6a0e46b nodeName:}" failed. No retries permitted until 2026-07-29 18:35:39.141270005 +0000 UTC m=+65.092837076 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret") pod "global-pull-secret-syncer-d9d6c" (UID: "32e483de-2fdb-429d-bb01-5f7ea6a0e46b") : object "kube-system"/"original-pull-secret" not registered
Jul 29 18:35:07.242136 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:07.242020    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs\") pod \"network-metrics-daemon-2vbxt\" (UID: \"ef2dbc84-9591-4b1b-8613-7887422229f2\") " pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:35:07.242325 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:07.242193    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 18:35:07.242325 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:07.242282    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs podName:ef2dbc84-9591-4b1b-8613-7887422229f2 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:39.242252244 +0000 UTC m=+65.193819335 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs") pod "network-metrics-daemon-2vbxt" (UID: "ef2dbc84-9591-4b1b-8613-7887422229f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 18:35:07.343349 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:07.343315    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsw6v\" (UniqueName: \"kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v\") pod \"network-check-target-2dp6b\" (UID: \"cbaacddf-394f-43c1-b02b-f85d651ebe28\") " pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:35:07.343567 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:07.343456    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 18:35:07.343567 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:07.343476    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 18:35:07.343567 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:07.343489    2579 projected.go:196] Error preparing data for projected volume kube-api-access-hsw6v for pod openshift-network-diagnostics/network-check-target-2dp6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 18:35:07.343746 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:07.343571    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v podName:cbaacddf-394f-43c1-b02b-f85d651ebe28 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:39.343556519 +0000 UTC m=+65.295123574 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hsw6v" (UniqueName: "kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v") pod "network-check-target-2dp6b" (UID: "cbaacddf-394f-43c1-b02b-f85d651ebe28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 18:35:07.579467 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:07.579434    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:35:07.579944 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:07.579577    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2vbxt" podUID="ef2dbc84-9591-4b1b-8613-7887422229f2"
Jul 29 18:35:08.580131 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.579921    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:35:08.580617 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.580001    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:35:08.580617 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:08.580243    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9d6c" podUID="32e483de-2fdb-429d-bb01-5f7ea6a0e46b"
Jul 29 18:35:08.580617 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:08.580318    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2dp6b" podUID="cbaacddf-394f-43c1-b02b-f85d651ebe28"
Jul 29 18:35:08.915626 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.915597    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeReady"
Jul 29 18:35:08.915781 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.915748    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 18:35:08.953058 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.953004    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d548586c6-6g7fk"]
Jul 29 18:35:08.954884 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.954864    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"]
Jul 29 18:35:08.955014 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.954997    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:08.956448 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.956422    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:08.957389 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.957054    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-766fd78c95-kbbsx"]
Jul 29 18:35:08.961414 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.961385    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"]
Jul 29 18:35:08.961538 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.961449    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:08.961538 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.961487    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 18:35:08.961631 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.961582    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-q6xm9\""
Jul 29 18:35:08.961677 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.961633    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v2s6m\""
Jul 29 18:35:08.961727 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.961702    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 18:35:08.962840 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.962822    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 18:35:08.962960 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.962943    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 18:35:08.963563 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.963544    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-pzkcg"]
Jul 29 18:35:08.963683 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.963643    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"
Jul 29 18:35:08.963934 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.963828    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 29 18:35:08.964130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.964115    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 29 18:35:08.964302 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.964288    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 18:35:08.965284 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.965265    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-f8wr7"]
Jul 29 18:35:08.965404 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.965385    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:08.966633 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.966617    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 18:35:08.966717 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.966702    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 29 18:35:08.966814 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.966799    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-wwht4"]
Jul 29 18:35:08.967222 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.966979    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:08.967222 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.966991    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 29 18:35:08.967222 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.967002    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rgfqx\""
Jul 29 18:35:08.967222 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.967017    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 18:35:08.967222 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.967003    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 29 18:35:08.967222 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.967131    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 29 18:35:08.967694 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.967680    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-xvlk7\""
Jul 29 18:35:08.967758 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.967730    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 29 18:35:08.967897 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.967883    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 29 18:35:08.967961 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.967938    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 29 18:35:08.968089 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.968075    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 18:35:08.968125 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.968090    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 29 18:35:08.968461 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.968443    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 18:35:08.968544 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.968478    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-tqpc9\""
Jul 29 18:35:08.968544 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.968490    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 29 18:35:08.968637 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.968555    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-wwht4"
Jul 29 18:35:08.968752 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.968726    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 18:35:08.974268 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.974243    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 29 18:35:08.974395 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.974329    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 29 18:35:08.974484 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.974468    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 29 18:35:08.974628 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.974609    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 29 18:35:08.974740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.974672    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 29 18:35:08.975105 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.975067    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"]
Jul 29 18:35:08.975358 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.975227    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-fffg7\""
Jul 29 18:35:08.975358 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.975268    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 29 18:35:08.975358 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.975295    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-lz62b\""
Jul 29 18:35:08.977643 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.977621    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-wwht4"]
Jul 29 18:35:08.979892 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.979668    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 29 18:35:08.982464 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.982442    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 29 18:35:08.983550 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.982672    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"]
Jul 29 18:35:08.983725 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.983667    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d548586c6-6g7fk"]
Jul 29 18:35:08.984322 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.984243    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-pzkcg"]
Jul 29 18:35:08.992530 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.992494    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6z9cl"]
Jul 29 18:35:08.994782 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.994761    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kdnwg"]
Jul 29 18:35:08.995348 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.995327    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6z9cl"
Jul 29 18:35:08.996904 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.996886    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-766fd78c95-kbbsx"]
Jul 29 18:35:08.996986 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.996912    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-f8wr7"]
Jul 29 18:35:08.996986 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.996977    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:08.999014 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.998993    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 18:35:08.999106 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.999040    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 18:35:08.999222 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.999207    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 18:35:08.999418 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.999392    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hgmvm\""
Jul 29 18:35:08.999418 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.999415    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 18:35:08.999727 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.999423    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q2mp4\""
Jul 29 18:35:08.999727 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:08.999442    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 18:35:09.009344 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.009320    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6z9cl"]
Jul 29 18:35:09.010046 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.010028    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kdnwg"]
Jul 29 18:35:09.055122 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055088    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:09.055292 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055130    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqpr7\" (UniqueName: \"kubernetes.io/projected/66f3a250-259a-4db3-8a53-bba41080d0cd-kube-api-access-kqpr7\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:09.055292 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-stats-auth\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:09.055292 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055169    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cb702a20-d5e5-4070-b422-649738adff18-ca-trust-extracted\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.055292 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055232    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1e7b645a-55d5-430f-a998-91b89c45b885-serving-cert\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.055292 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1e7b645a-55d5-430f-a998-91b89c45b885-snapshots\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.055483 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055302    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/272b03c5-141c-4a65-b707-e1d547ceab87-serving-cert\") pod \"console-operator-584dcb6b54-f8wr7\" (UID: \"272b03c5-141c-4a65-b707-e1d547ceab87\") " pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:09.055483 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055322    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-796hc\" (UniqueName: \"kubernetes.io/projected/1e7b645a-55d5-430f-a998-91b89c45b885-kube-api-access-796hc\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.055483 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055341    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-924xb\" (UniqueName: \"kubernetes.io/projected/0dd1da74-2c12-4022-be1e-3b0879d4c0ba-kube-api-access-924xb\") pod \"ingress-canary-6z9cl\" (UID: \"0dd1da74-2c12-4022-be1e-3b0879d4c0ba\") " pod="openshift-ingress-canary/ingress-canary-6z9cl"
Jul 29 18:35:09.055483 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055364    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj462\" (UniqueName: \"kubernetes.io/projected/33d38e5d-13d5-45d9-bc61-fb9ba537556b-kube-api-access-qj462\") pod \"volume-data-source-validator-549c6d54cf-wwht4\" (UID: \"33d38e5d-13d5-45d9-bc61-fb9ba537556b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-wwht4"
Jul 29 18:35:09.055483 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055396    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gzvjm\" (UID: \"aedf1ecb-e610-4bf3-9688-a969c14dd65b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:09.055483 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055413    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj6nr\" (UniqueName: \"kubernetes.io/projected/abbf5f2c-044a-4007-9656-54b2b1c454fb-kube-api-access-nj6nr\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:09.055483 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055450    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0dd1da74-2c12-4022-be1e-3b0879d4c0ba-cert\") pod \"ingress-canary-6z9cl\" (UID: \"0dd1da74-2c12-4022-be1e-3b0879d4c0ba\") " pod="openshift-ingress-canary/ingress-canary-6z9cl"
Jul 29 18:35:09.055483 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055482    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.055824 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055524    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:09.055824 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055542    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/272b03c5-141c-4a65-b707-e1d547ceab87-config\") pod \"console-operator-584dcb6b54-f8wr7\" (UID: \"272b03c5-141c-4a65-b707-e1d547ceab87\") " pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:09.055824 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055558    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-bound-sa-token\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.055824 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055598    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cb702a20-d5e5-4070-b422-649738adff18-image-registry-private-configuration\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.055824 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055617    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcv2m\" (UniqueName: \"kubernetes.io/projected/aedf1ecb-e610-4bf3-9688-a969c14dd65b-kube-api-access-rcv2m\") pod \"cluster-monitoring-operator-548b489cb8-gzvjm\" (UID: \"aedf1ecb-e610-4bf3-9688-a969c14dd65b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:09.055824 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055640    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cb702a20-d5e5-4070-b422-649738adff18-registry-certificates\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.055824 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055656    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aedf1ecb-e610-4bf3-9688-a969c14dd65b-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-gzvjm\" (UID: \"aedf1ecb-e610-4bf3-9688-a969c14dd65b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:09.055824 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055673    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e7b645a-55d5-430f-a998-91b89c45b885-service-ca-bundle\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.055824 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055690    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-xrcrz\" (UID: \"fb928e9d-7a00-471f-ae5e-95524175cb81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"
Jul 29 18:35:09.055824 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055710    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cb702a20-d5e5-4070-b422-649738adff18-installation-pull-secrets\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.055824 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055742    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssp5w\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-kube-api-access-ssp5w\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.055824 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055775    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-default-certificate\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:09.055824 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055805    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/272b03c5-141c-4a65-b707-e1d547ceab87-trusted-ca\") pod \"console-operator-584dcb6b54-f8wr7\" (UID: \"272b03c5-141c-4a65-b707-e1d547ceab87\") " pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:09.055824 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055829    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:09.056274 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055851    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e7b645a-55d5-430f-a998-91b89c45b885-tmp\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.056274 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055874    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/66f3a250-259a-4db3-8a53-bba41080d0cd-config-volume\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:09.056274 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055900    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98tmw\" (UniqueName: \"kubernetes.io/projected/272b03c5-141c-4a65-b707-e1d547ceab87-kube-api-access-98tmw\") pod \"console-operator-584dcb6b54-f8wr7\" (UID: \"272b03c5-141c-4a65-b707-e1d547ceab87\") " pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:09.056274 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055924    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrtv2\" (UniqueName: \"kubernetes.io/projected/fb928e9d-7a00-471f-ae5e-95524175cb81-kube-api-access-hrtv2\") pod \"cluster-samples-operator-6748489656-xrcrz\" (UID: \"fb928e9d-7a00-471f-ae5e-95524175cb81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"
Jul 29 18:35:09.056274 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66f3a250-259a-4db3-8a53-bba41080d0cd-tmp-dir\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:09.056274 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055968    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb702a20-d5e5-4070-b422-649738adff18-trusted-ca\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.056274 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.055982    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e7b645a-55d5-430f-a998-91b89c45b885-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.156539 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.156445    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1e7b645a-55d5-430f-a998-91b89c45b885-serving-cert\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.156539 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.156485    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1e7b645a-55d5-430f-a998-91b89c45b885-snapshots\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.156539 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.156515    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/272b03c5-141c-4a65-b707-e1d547ceab87-serving-cert\") pod \"console-operator-584dcb6b54-f8wr7\" (UID: \"272b03c5-141c-4a65-b707-e1d547ceab87\") " pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:09.156805 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.156641    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-796hc\" (UniqueName: \"kubernetes.io/projected/1e7b645a-55d5-430f-a998-91b89c45b885-kube-api-access-796hc\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.156805 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.156695    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-924xb\" (UniqueName: \"kubernetes.io/projected/0dd1da74-2c12-4022-be1e-3b0879d4c0ba-kube-api-access-924xb\") pod \"ingress-canary-6z9cl\" (UID: \"0dd1da74-2c12-4022-be1e-3b0879d4c0ba\") " pod="openshift-ingress-canary/ingress-canary-6z9cl"
Jul 29 18:35:09.156805 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.156725    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj462\" (UniqueName: \"kubernetes.io/projected/33d38e5d-13d5-45d9-bc61-fb9ba537556b-kube-api-access-qj462\") pod \"volume-data-source-validator-549c6d54cf-wwht4\" (UID: \"33d38e5d-13d5-45d9-bc61-fb9ba537556b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-wwht4"
Jul 29 18:35:09.156805 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.156770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gzvjm\" (UID: \"aedf1ecb-e610-4bf3-9688-a969c14dd65b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:09.157011 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.156812    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj6nr\" (UniqueName: \"kubernetes.io/projected/abbf5f2c-044a-4007-9656-54b2b1c454fb-kube-api-access-nj6nr\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:09.157011 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.156838    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0dd1da74-2c12-4022-be1e-3b0879d4c0ba-cert\") pod \"ingress-canary-6z9cl\" (UID: \"0dd1da74-2c12-4022-be1e-3b0879d4c0ba\") " pod="openshift-ingress-canary/ingress-canary-6z9cl"
Jul 29 18:35:09.157011 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.156922    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 18:35:09.157011 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.156989    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls podName:aedf1ecb-e610-4bf3-9688-a969c14dd65b nodeName:}" failed. No retries permitted until 2026-07-29 18:35:09.656968214 +0000 UTC m=+35.608535269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-gzvjm" (UID: "aedf1ecb-e610-4bf3-9688-a969c14dd65b") : secret "cluster-monitoring-operator-tls" not found
Jul 29 18:35:09.157322 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157117    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1e7b645a-55d5-430f-a998-91b89c45b885-snapshots\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.157322 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157143    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.157322 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157177    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:09.157322 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157209    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/272b03c5-141c-4a65-b707-e1d547ceab87-config\") pod \"console-operator-584dcb6b54-f8wr7\" (UID: \"272b03c5-141c-4a65-b707-e1d547ceab87\") " pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:09.157322 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.157219    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 18:35:09.157322 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.157232    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d548586c6-6g7fk: secret "image-registry-tls" not found
Jul 29 18:35:09.157322 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-bound-sa-token\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.157322 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.157271    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls podName:cb702a20-d5e5-4070-b422-649738adff18 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:09.657256315 +0000 UTC m=+35.608823375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls") pod "image-registry-6d548586c6-6g7fk" (UID: "cb702a20-d5e5-4070-b422-649738adff18") : secret "image-registry-tls" not found
Jul 29 18:35:09.157322 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157288    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cb702a20-d5e5-4070-b422-649738adff18-image-registry-private-configuration\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.157322 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.157314    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle podName:abbf5f2c-044a-4007-9656-54b2b1c454fb nodeName:}" failed. No retries permitted until 2026-07-29 18:35:09.657295772 +0000 UTC m=+35.608862846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle") pod "router-default-766fd78c95-kbbsx" (UID: "abbf5f2c-044a-4007-9656-54b2b1c454fb") : configmap references non-existent config key: service-ca.crt
Jul 29 18:35:09.157973 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157350    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcv2m\" (UniqueName: \"kubernetes.io/projected/aedf1ecb-e610-4bf3-9688-a969c14dd65b-kube-api-access-rcv2m\") pod \"cluster-monitoring-operator-548b489cb8-gzvjm\" (UID: \"aedf1ecb-e610-4bf3-9688-a969c14dd65b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:09.157973 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157388    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cb702a20-d5e5-4070-b422-649738adff18-registry-certificates\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.157973 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aedf1ecb-e610-4bf3-9688-a969c14dd65b-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-gzvjm\" (UID: \"aedf1ecb-e610-4bf3-9688-a969c14dd65b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:09.157973 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157430    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e7b645a-55d5-430f-a998-91b89c45b885-service-ca-bundle\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.157973 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157457    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-xrcrz\" (UID: \"fb928e9d-7a00-471f-ae5e-95524175cb81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"
Jul 29 18:35:09.157973 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157484    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cb702a20-d5e5-4070-b422-649738adff18-installation-pull-secrets\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.157973 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssp5w\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-kube-api-access-ssp5w\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.157973 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157549    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-default-certificate\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:09.157973 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157573    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/272b03c5-141c-4a65-b707-e1d547ceab87-trusted-ca\") pod \"console-operator-584dcb6b54-f8wr7\" (UID: \"272b03c5-141c-4a65-b707-e1d547ceab87\") " pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:09.157973 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157594    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:09.157973 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157616    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e7b645a-55d5-430f-a998-91b89c45b885-tmp\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.157973 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157638    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/66f3a250-259a-4db3-8a53-bba41080d0cd-config-volume\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:09.157973 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157665    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98tmw\" (UniqueName: \"kubernetes.io/projected/272b03c5-141c-4a65-b707-e1d547ceab87-kube-api-access-98tmw\") pod \"console-operator-584dcb6b54-f8wr7\" (UID: \"272b03c5-141c-4a65-b707-e1d547ceab87\") " pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:09.157973 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157687    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrtv2\" (UniqueName: \"kubernetes.io/projected/fb928e9d-7a00-471f-ae5e-95524175cb81-kube-api-access-hrtv2\") pod \"cluster-samples-operator-6748489656-xrcrz\" (UID: \"fb928e9d-7a00-471f-ae5e-95524175cb81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"
Jul 29 18:35:09.157973 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66f3a250-259a-4db3-8a53-bba41080d0cd-tmp-dir\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:09.157973 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157743    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb702a20-d5e5-4070-b422-649738adff18-trusted-ca\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.158731 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157768    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e7b645a-55d5-430f-a998-91b89c45b885-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.158731 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157800    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:09.158731 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157837    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqpr7\" (UniqueName: \"kubernetes.io/projected/66f3a250-259a-4db3-8a53-bba41080d0cd-kube-api-access-kqpr7\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:09.158731 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157838    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/272b03c5-141c-4a65-b707-e1d547ceab87-config\") pod \"console-operator-584dcb6b54-f8wr7\" (UID: \"272b03c5-141c-4a65-b707-e1d547ceab87\") " pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:09.158731 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-stats-auth\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:09.158731 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157891    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cb702a20-d5e5-4070-b422-649738adff18-ca-trust-extracted\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.158731 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.157951    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cb702a20-d5e5-4070-b422-649738adff18-registry-certificates\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.158731 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.158200    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e7b645a-55d5-430f-a998-91b89c45b885-tmp\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.158731 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.158222    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cb702a20-d5e5-4070-b422-649738adff18-ca-trust-extracted\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.159270 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.158990    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/66f3a250-259a-4db3-8a53-bba41080d0cd-config-volume\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:09.159978 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.159650    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66f3a250-259a-4db3-8a53-bba41080d0cd-tmp-dir\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:09.160465 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.160445    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/272b03c5-141c-4a65-b707-e1d547ceab87-trusted-ca\") pod \"console-operator-584dcb6b54-f8wr7\" (UID: \"272b03c5-141c-4a65-b707-e1d547ceab87\") " pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:09.160940 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.160672    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aedf1ecb-e610-4bf3-9688-a969c14dd65b-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-gzvjm\" (UID: \"aedf1ecb-e610-4bf3-9688-a969c14dd65b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:09.160940 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.160678    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb702a20-d5e5-4070-b422-649738adff18-trusted-ca\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.161132 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.161116    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 18:35:09.161266 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.161255    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs podName:abbf5f2c-044a-4007-9656-54b2b1c454fb nodeName:}" failed. No retries permitted until 2026-07-29 18:35:09.661235418 +0000 UTC m=+35.612802474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs") pod "router-default-766fd78c95-kbbsx" (UID: "abbf5f2c-044a-4007-9656-54b2b1c454fb") : secret "router-metrics-certs-default" not found
Jul 29 18:35:09.162059 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.161901    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cb702a20-d5e5-4070-b422-649738adff18-installation-pull-secrets\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.162059 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.161933    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-default-certificate\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:09.162059 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.161992    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cb702a20-d5e5-4070-b422-649738adff18-image-registry-private-configuration\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.162059 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.162021    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 18:35:09.162059 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.162028    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0dd1da74-2c12-4022-be1e-3b0879d4c0ba-cert\") pod \"ingress-canary-6z9cl\" (UID: \"0dd1da74-2c12-4022-be1e-3b0879d4c0ba\") " pod="openshift-ingress-canary/ingress-canary-6z9cl"
Jul 29 18:35:09.162395 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.162068    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls podName:fb928e9d-7a00-471f-ae5e-95524175cb81 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:09.662053035 +0000 UTC m=+35.613620089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls") pod "cluster-samples-operator-6748489656-xrcrz" (UID: "fb928e9d-7a00-471f-ae5e-95524175cb81") : secret "samples-operator-tls" not found
Jul 29 18:35:09.162395 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.162263    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/272b03c5-141c-4a65-b707-e1d547ceab87-serving-cert\") pod \"console-operator-584dcb6b54-f8wr7\" (UID: \"272b03c5-141c-4a65-b707-e1d547ceab87\") " pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:09.162523 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.162472    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e7b645a-55d5-430f-a998-91b89c45b885-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.162743 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.162710    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 18:35:09.162841 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.162787    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e7b645a-55d5-430f-a998-91b89c45b885-service-ca-bundle\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.163004 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.162958    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1e7b645a-55d5-430f-a998-91b89c45b885-serving-cert\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.163112 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.163031    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls podName:66f3a250-259a-4db3-8a53-bba41080d0cd nodeName:}" failed. No retries permitted until 2026-07-29 18:35:09.662972871 +0000 UTC m=+35.614539939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls") pod "dns-default-kdnwg" (UID: "66f3a250-259a-4db3-8a53-bba41080d0cd") : secret "dns-default-metrics-tls" not found
Jul 29 18:35:09.166201 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.165963    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-stats-auth\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:09.169194 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.169166    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-bound-sa-token\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.170633 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.170051    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqpr7\" (UniqueName: \"kubernetes.io/projected/66f3a250-259a-4db3-8a53-bba41080d0cd-kube-api-access-kqpr7\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:09.170633 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.170164    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj462\" (UniqueName: \"kubernetes.io/projected/33d38e5d-13d5-45d9-bc61-fb9ba537556b-kube-api-access-qj462\") pod \"volume-data-source-validator-549c6d54cf-wwht4\" (UID: \"33d38e5d-13d5-45d9-bc61-fb9ba537556b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-wwht4"
Jul 29 18:35:09.170633 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.170166    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-924xb\" (UniqueName: \"kubernetes.io/projected/0dd1da74-2c12-4022-be1e-3b0879d4c0ba-kube-api-access-924xb\") pod \"ingress-canary-6z9cl\" (UID: \"0dd1da74-2c12-4022-be1e-3b0879d4c0ba\") " pod="openshift-ingress-canary/ingress-canary-6z9cl"
Jul 29 18:35:09.170633 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.170482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98tmw\" (UniqueName: \"kubernetes.io/projected/272b03c5-141c-4a65-b707-e1d547ceab87-kube-api-access-98tmw\") pod \"console-operator-584dcb6b54-f8wr7\" (UID: \"272b03c5-141c-4a65-b707-e1d547ceab87\") " pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:09.170926 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.170651    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssp5w\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-kube-api-access-ssp5w\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.170926 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.170706    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj6nr\" (UniqueName: \"kubernetes.io/projected/abbf5f2c-044a-4007-9656-54b2b1c454fb-kube-api-access-nj6nr\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:09.171178 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.171158    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcv2m\" (UniqueName: \"kubernetes.io/projected/aedf1ecb-e610-4bf3-9688-a969c14dd65b-kube-api-access-rcv2m\") pod \"cluster-monitoring-operator-548b489cb8-gzvjm\" (UID: \"aedf1ecb-e610-4bf3-9688-a969c14dd65b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:09.171178 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.171173    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-796hc\" (UniqueName: \"kubernetes.io/projected/1e7b645a-55d5-430f-a998-91b89c45b885-kube-api-access-796hc\") pod \"insights-operator-5885fdb759-pzkcg\" (UID: \"1e7b645a-55d5-430f-a998-91b89c45b885\") " pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.171486 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.171464    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrtv2\" (UniqueName: \"kubernetes.io/projected/fb928e9d-7a00-471f-ae5e-95524175cb81-kube-api-access-hrtv2\") pod \"cluster-samples-operator-6748489656-xrcrz\" (UID: \"fb928e9d-7a00-471f-ae5e-95524175cb81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"
Jul 29 18:35:09.311194 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.311166    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-pzkcg"
Jul 29 18:35:09.318312 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.318288    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:09.323523 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.323490    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-wwht4"
Jul 29 18:35:09.329950 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.329924    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6z9cl"
Jul 29 18:35:09.505246 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.505218    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-wwht4"]
Jul 29 18:35:09.508359 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:35:09.508207    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d38e5d_13d5_45d9_bc61_fb9ba537556b.slice/crio-5395ab3aa781db3cd5438e9b23c1cba45e5d09075939183572963e990fcff2d7 WatchSource:0}: Error finding container 5395ab3aa781db3cd5438e9b23c1cba45e5d09075939183572963e990fcff2d7: Status 404 returned error can't find the container with id 5395ab3aa781db3cd5438e9b23c1cba45e5d09075939183572963e990fcff2d7
Jul 29 18:35:09.512222 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.512201    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6z9cl"]
Jul 29 18:35:09.515266 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:35:09.515243    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dd1da74_2c12_4022_be1e_3b0879d4c0ba.slice/crio-dd3a84c6123ad7a70e9d71928806e864a1deabade6f970201c55273aabdb9e82 WatchSource:0}: Error finding container dd3a84c6123ad7a70e9d71928806e864a1deabade6f970201c55273aabdb9e82: Status 404 returned error can't find the container with id dd3a84c6123ad7a70e9d71928806e864a1deabade6f970201c55273aabdb9e82
Jul 29 18:35:09.526335 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.526311    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-f8wr7"]
Jul 29 18:35:09.528266 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:35:09.528244    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod272b03c5_141c_4a65_b707_e1d547ceab87.slice/crio-450cc940cc90ea2261bd95b34fb0840ec042602a3c32a267dcb6575c9137dfd8 WatchSource:0}: Error finding container 450cc940cc90ea2261bd95b34fb0840ec042602a3c32a267dcb6575c9137dfd8: Status 404 returned error can't find the container with id 450cc940cc90ea2261bd95b34fb0840ec042602a3c32a267dcb6575c9137dfd8
Jul 29 18:35:09.530423 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.530402    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-pzkcg"]
Jul 29 18:35:09.532814 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:35:09.532795    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e7b645a_55d5_430f_a998_91b89c45b885.slice/crio-a1a1854690c13827f34d506dd281f4ca2e83970716ea60325cae60dd06e6e025 WatchSource:0}: Error finding container a1a1854690c13827f34d506dd281f4ca2e83970716ea60325cae60dd06e6e025: Status 404 returned error can't find the container with id a1a1854690c13827f34d506dd281f4ca2e83970716ea60325cae60dd06e6e025
Jul 29 18:35:09.579612 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.579580    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:35:09.582897 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.582880    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 18:35:09.583469 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.583042    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g7g2r\""
Jul 29 18:35:09.663729 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.663695    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:09.663916 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.663769    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gzvjm\" (UID: \"aedf1ecb-e610-4bf3-9688-a969c14dd65b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:09.663916 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.663838    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:09.663916 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.663860    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 18:35:09.663916 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.663879    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:09.663916 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.663906    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls podName:aedf1ecb-e610-4bf3-9688-a969c14dd65b nodeName:}" failed. No retries permitted until 2026-07-29 18:35:10.663892555 +0000 UTC m=+36.615459611 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-gzvjm" (UID: "aedf1ecb-e610-4bf3-9688-a969c14dd65b") : secret "cluster-monitoring-operator-tls" not found
Jul 29 18:35:09.664198 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.663943    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-xrcrz\" (UID: \"fb928e9d-7a00-471f-ae5e-95524175cb81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"
Jul 29 18:35:09.664198 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.663974    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle podName:abbf5f2c-044a-4007-9656-54b2b1c454fb nodeName:}" failed. No retries permitted until 2026-07-29 18:35:10.663960322 +0000 UTC m=+36.615527377 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle") pod "router-default-766fd78c95-kbbsx" (UID: "abbf5f2c-044a-4007-9656-54b2b1c454fb") : configmap references non-existent config key: service-ca.crt
Jul 29 18:35:09.664198 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.663994    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 18:35:09.664198 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.664007    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 18:35:09.664198 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.664013    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d548586c6-6g7fk: secret "image-registry-tls" not found
Jul 29 18:35:09.664198 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.664016    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:09.664198 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.664044    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls podName:fb928e9d-7a00-471f-ae5e-95524175cb81 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:10.664033823 +0000 UTC m=+36.615600885 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls") pod "cluster-samples-operator-6748489656-xrcrz" (UID: "fb928e9d-7a00-471f-ae5e-95524175cb81") : secret "samples-operator-tls" not found
Jul 29 18:35:09.664198 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.664058    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 18:35:09.664198 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.664064    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls podName:cb702a20-d5e5-4070-b422-649738adff18 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:10.664055064 +0000 UTC m=+36.615622125 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls") pod "image-registry-6d548586c6-6g7fk" (UID: "cb702a20-d5e5-4070-b422-649738adff18") : secret "image-registry-tls" not found
Jul 29 18:35:09.664198 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.664064    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 18:35:09.664198 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.664097    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls podName:66f3a250-259a-4db3-8a53-bba41080d0cd nodeName:}" failed. No retries permitted until 2026-07-29 18:35:10.66408585 +0000 UTC m=+36.615652909 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls") pod "dns-default-kdnwg" (UID: "66f3a250-259a-4db3-8a53-bba41080d0cd") : secret "dns-default-metrics-tls" not found
Jul 29 18:35:09.664198 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:09.664136    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs podName:abbf5f2c-044a-4007-9656-54b2b1c454fb nodeName:}" failed. No retries permitted until 2026-07-29 18:35:10.66412415 +0000 UTC m=+36.615691208 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs") pod "router-default-766fd78c95-kbbsx" (UID: "abbf5f2c-044a-4007-9656-54b2b1c454fb") : secret "router-metrics-certs-default" not found
Jul 29 18:35:09.801482 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.801451    2579 generic.go:356] "Generic (PLEG): container finished" podID="98679e95-83f4-48b6-8295-f097eebbd6bb" containerID="dfcd52af515d059628343fb8c4ec9c2825176c570d2248c67ff6e2668b3628e5" exitCode=0
Jul 29 18:35:09.801660 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.801546    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2kjtm" event={"ID":"98679e95-83f4-48b6-8295-f097eebbd6bb","Type":"ContainerDied","Data":"dfcd52af515d059628343fb8c4ec9c2825176c570d2248c67ff6e2668b3628e5"}
Jul 29 18:35:09.802697 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.802679    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7" event={"ID":"272b03c5-141c-4a65-b707-e1d547ceab87","Type":"ContainerStarted","Data":"450cc940cc90ea2261bd95b34fb0840ec042602a3c32a267dcb6575c9137dfd8"}
Jul 29 18:35:09.803714 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.803692    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-wwht4" event={"ID":"33d38e5d-13d5-45d9-bc61-fb9ba537556b","Type":"ContainerStarted","Data":"5395ab3aa781db3cd5438e9b23c1cba45e5d09075939183572963e990fcff2d7"}
Jul 29 18:35:09.804743 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.804726    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6z9cl" event={"ID":"0dd1da74-2c12-4022-be1e-3b0879d4c0ba","Type":"ContainerStarted","Data":"dd3a84c6123ad7a70e9d71928806e864a1deabade6f970201c55273aabdb9e82"}
Jul 29 18:35:09.805842 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:09.805824    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-pzkcg" event={"ID":"1e7b645a-55d5-430f-a998-91b89c45b885","Type":"ContainerStarted","Data":"a1a1854690c13827f34d506dd281f4ca2e83970716ea60325cae60dd06e6e025"}
Jul 29 18:35:10.579021 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:10.578941    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:35:10.579728 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:10.579262    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:35:10.581848 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:10.581824    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b24qk\""
Jul 29 18:35:10.581952 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:10.581825    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 18:35:10.581952 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:10.581915    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 18:35:10.582072 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:10.581829    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 18:35:10.677754 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:10.677717    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:10.678319 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:10.677842    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gzvjm\" (UID: \"aedf1ecb-e610-4bf3-9688-a969c14dd65b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:10.678319 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:10.677867    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:10.678319 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:10.677903    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 18:35:10.678319 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:10.677918    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:10.678319 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:10.677983    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls podName:66f3a250-259a-4db3-8a53-bba41080d0cd nodeName:}" failed. No retries permitted until 2026-07-29 18:35:12.677962247 +0000 UTC m=+38.629529301 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls") pod "dns-default-kdnwg" (UID: "66f3a250-259a-4db3-8a53-bba41080d0cd") : secret "dns-default-metrics-tls" not found
Jul 29 18:35:10.678319 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:10.678033    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-xrcrz\" (UID: \"fb928e9d-7a00-471f-ae5e-95524175cb81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"
Jul 29 18:35:10.678319 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:10.678079    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:10.678319 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:10.678098    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle podName:abbf5f2c-044a-4007-9656-54b2b1c454fb nodeName:}" failed. No retries permitted until 2026-07-29 18:35:12.678070692 +0000 UTC m=+38.629637746 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle") pod "router-default-766fd78c95-kbbsx" (UID: "abbf5f2c-044a-4007-9656-54b2b1c454fb") : configmap references non-existent config key: service-ca.crt
Jul 29 18:35:10.678319 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:10.678151    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 18:35:10.678319 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:10.678188    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs podName:abbf5f2c-044a-4007-9656-54b2b1c454fb nodeName:}" failed. No retries permitted until 2026-07-29 18:35:12.678176045 +0000 UTC m=+38.629743100 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs") pod "router-default-766fd78c95-kbbsx" (UID: "abbf5f2c-044a-4007-9656-54b2b1c454fb") : secret "router-metrics-certs-default" not found
Jul 29 18:35:10.678319 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:10.678239    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 18:35:10.678923 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:10.678337    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls podName:aedf1ecb-e610-4bf3-9688-a969c14dd65b nodeName:}" failed. No retries permitted until 2026-07-29 18:35:12.678313612 +0000 UTC m=+38.629880666 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-gzvjm" (UID: "aedf1ecb-e610-4bf3-9688-a969c14dd65b") : secret "cluster-monitoring-operator-tls" not found
Jul 29 18:35:10.678923 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:10.678428    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 18:35:10.678923 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:10.678438    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d548586c6-6g7fk: secret "image-registry-tls" not found
Jul 29 18:35:10.678923 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:10.678461    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls podName:cb702a20-d5e5-4070-b422-649738adff18 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:12.678452642 +0000 UTC m=+38.630019698 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls") pod "image-registry-6d548586c6-6g7fk" (UID: "cb702a20-d5e5-4070-b422-649738adff18") : secret "image-registry-tls" not found
Jul 29 18:35:10.678923 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:10.678553    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 18:35:10.678923 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:10.678602    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls podName:fb928e9d-7a00-471f-ae5e-95524175cb81 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:12.678594429 +0000 UTC m=+38.630161485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls") pod "cluster-samples-operator-6748489656-xrcrz" (UID: "fb928e9d-7a00-471f-ae5e-95524175cb81") : secret "samples-operator-tls" not found
Jul 29 18:35:10.814330 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:10.814160    2579 generic.go:356] "Generic (PLEG): container finished" podID="98679e95-83f4-48b6-8295-f097eebbd6bb" containerID="780815e7ac5b172c0d32d01ac05c0dcd37f6d387c6f2a5c260a23813c37ed6fd" exitCode=0
Jul 29 18:35:10.814330 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:10.814235    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2kjtm" event={"ID":"98679e95-83f4-48b6-8295-f097eebbd6bb","Type":"ContainerDied","Data":"780815e7ac5b172c0d32d01ac05c0dcd37f6d387c6f2a5c260a23813c37ed6fd"}
Jul 29 18:35:12.698392 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:12.698081    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:12.698392 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:12.698365    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:12.699010 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:12.698251    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 18:35:12.699010 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:12.698427    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d548586c6-6g7fk: secret "image-registry-tls" not found
Jul 29 18:35:12.699010 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:12.698516    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls podName:cb702a20-d5e5-4070-b422-649738adff18 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:16.698470685 +0000 UTC m=+42.650037759 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls") pod "image-registry-6d548586c6-6g7fk" (UID: "cb702a20-d5e5-4070-b422-649738adff18") : secret "image-registry-tls" not found
Jul 29 18:35:12.699010 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:12.698423    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-xrcrz\" (UID: \"fb928e9d-7a00-471f-ae5e-95524175cb81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"
Jul 29 18:35:12.699010 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:12.698524    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 18:35:12.699010 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:12.698537    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle podName:abbf5f2c-044a-4007-9656-54b2b1c454fb nodeName:}" failed. No retries permitted until 2026-07-29 18:35:16.698527209 +0000 UTC m=+42.650094269 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle") pod "router-default-766fd78c95-kbbsx" (UID: "abbf5f2c-044a-4007-9656-54b2b1c454fb") : configmap references non-existent config key: service-ca.crt
Jul 29 18:35:12.699010 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:12.698576    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls podName:fb928e9d-7a00-471f-ae5e-95524175cb81 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:16.698558795 +0000 UTC m=+42.650125863 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls") pod "cluster-samples-operator-6748489656-xrcrz" (UID: "fb928e9d-7a00-471f-ae5e-95524175cb81") : secret "samples-operator-tls" not found
Jul 29 18:35:12.699010 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:12.698619    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:12.699010 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:12.698672    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:12.699010 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:12.698763    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 18:35:12.699010 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:12.698804    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs podName:abbf5f2c-044a-4007-9656-54b2b1c454fb nodeName:}" failed. No retries permitted until 2026-07-29 18:35:16.698791802 +0000 UTC m=+42.650358860 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs") pod "router-default-766fd78c95-kbbsx" (UID: "abbf5f2c-044a-4007-9656-54b2b1c454fb") : secret "router-metrics-certs-default" not found
Jul 29 18:35:12.699010 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:12.698762    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gzvjm\" (UID: \"aedf1ecb-e610-4bf3-9688-a969c14dd65b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:12.699010 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:12.698849    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 18:35:12.699010 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:12.698890    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls podName:aedf1ecb-e610-4bf3-9688-a969c14dd65b nodeName:}" failed. No retries permitted until 2026-07-29 18:35:16.698877698 +0000 UTC m=+42.650444767 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-gzvjm" (UID: "aedf1ecb-e610-4bf3-9688-a969c14dd65b") : secret "cluster-monitoring-operator-tls" not found
Jul 29 18:35:12.699010 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:12.698898    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 18:35:12.700733 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:12.698929    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls podName:66f3a250-259a-4db3-8a53-bba41080d0cd nodeName:}" failed. No retries permitted until 2026-07-29 18:35:16.698918786 +0000 UTC m=+42.650485844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls") pod "dns-default-kdnwg" (UID: "66f3a250-259a-4db3-8a53-bba41080d0cd") : secret "dns-default-metrics-tls" not found
Jul 29 18:35:12.821862 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:12.821827    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6z9cl" event={"ID":"0dd1da74-2c12-4022-be1e-3b0879d4c0ba","Type":"ContainerStarted","Data":"30f2046a51135e78f0f23f3106890a8cb073a4d5d2406bcbe89a8f0db018f576"}
Jul 29 18:35:12.823687 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:12.823658    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-pzkcg" event={"ID":"1e7b645a-55d5-430f-a998-91b89c45b885","Type":"ContainerStarted","Data":"052d8f58a8158f41e37feeff8ea57e8956f336fe89e4c7718cd315ed448375c1"}
Jul 29 18:35:12.827658 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:12.827624    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2kjtm" event={"ID":"98679e95-83f4-48b6-8295-f097eebbd6bb","Type":"ContainerStarted","Data":"cf27d4b4161afcfc9a4f69f24d49ca4fcdecd15bd2b1f3f6575d6edf986cd8c3"}
Jul 29 18:35:12.838844 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:12.838745    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6z9cl" podStartSLOduration=2.258754736 podStartE2EDuration="4.838723863s" totalImagesPullingTime="2.579969127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:12.838085936 +0000 UTC m=+38.789653014" watchObservedRunningTime="2026-07-29 18:35:12.838723863 +0000 UTC m=+38.790290942"
Jul 29 18:35:12.861084 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:12.861030    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2kjtm" podStartSLOduration=9.359567091 podStartE2EDuration="38.861009885s" totalImagesPullingTime="29.501442794s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 18:34:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:12.859792036 +0000 UTC m=+38.811359114" watchObservedRunningTime="2026-07-29 18:35:12.861009885 +0000 UTC m=+38.812576962"
Jul 29 18:35:12.878171 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:12.878101    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-pzkcg" podStartSLOduration=4.321830851 podStartE2EDuration="6.878079892s" totalImagesPullingTime="2.556249041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:12.877310192 +0000 UTC m=+38.828877293" watchObservedRunningTime="2026-07-29 18:35:12.878079892 +0000 UTC m=+38.829646971"
Jul 29 18:35:13.687600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.687497    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cc6bt"]
Jul 29 18:35:13.720113 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.720083    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cc6bt"]
Jul 29 18:35:13.720961 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.720238    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:13.723139 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.723115    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 18:35:13.723139 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.723115    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 18:35:13.723655 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.723617    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8px9m\""
Jul 29 18:35:13.811310 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.811270    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b0cbd483-407c-4f29-a033-fa0beda9bba7-crio-socket\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:13.811310 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.811316    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h7tz\" (UniqueName: \"kubernetes.io/projected/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-api-access-2h7tz\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:13.811592 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.811473    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b0cbd483-407c-4f29-a033-fa0beda9bba7-data-volume\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:13.811592 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.811529    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:13.811701 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.811657    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:13.832605 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.832437    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-wwht4" event={"ID":"33d38e5d-13d5-45d9-bc61-fb9ba537556b","Type":"ContainerStarted","Data":"1e294c61ba2f027430f410fb53d66450f9f60c5d0a11b9ea1825bbca176878be"}
Jul 29 18:35:13.849518 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.849460    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-wwht4" podStartSLOduration=4.577323084 podStartE2EDuration="7.849438025s" totalImagesPullingTime="3.272114941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:13.84831913 +0000 UTC m=+39.799886207" watchObservedRunningTime="2026-07-29 18:35:13.849438025 +0000 UTC m=+39.801005101"
Jul 29 18:35:13.912243 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.912200    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b0cbd483-407c-4f29-a033-fa0beda9bba7-data-volume\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:13.912243 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.912242    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:13.912527 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:13.912316    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 18:35:13.912527 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:13.912367    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm podName:b0cbd483-407c-4f29-a033-fa0beda9bba7 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:14.412354708 +0000 UTC m=+40.363921763 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cc6bt" (UID: "b0cbd483-407c-4f29-a033-fa0beda9bba7") : configmap "kube-rbac-proxy" not found
Jul 29 18:35:13.912527 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.912492    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:13.912707 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.912555    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b0cbd483-407c-4f29-a033-fa0beda9bba7-data-volume\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:13.912707 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:13.912658    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 18:35:13.912820 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.912704    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b0cbd483-407c-4f29-a033-fa0beda9bba7-crio-socket\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:13.912820 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.912740    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h7tz\" (UniqueName: \"kubernetes.io/projected/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-api-access-2h7tz\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:13.912820 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.912788    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b0cbd483-407c-4f29-a033-fa0beda9bba7-crio-socket\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:13.912967 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:13.912847    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls podName:b0cbd483-407c-4f29-a033-fa0beda9bba7 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:14.412816568 +0000 UTC m=+40.364383628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cc6bt" (UID: "b0cbd483-407c-4f29-a033-fa0beda9bba7") : secret "insights-runtime-extractor-tls" not found
Jul 29 18:35:13.923848 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:13.923820    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h7tz\" (UniqueName: \"kubernetes.io/projected/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-api-access-2h7tz\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:14.417904 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:14.417872    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:14.418063 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:14.417979    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:14.418063 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:14.418029    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 18:35:14.418141 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:14.418076    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 18:35:14.418141 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:14.418106    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls podName:b0cbd483-407c-4f29-a033-fa0beda9bba7 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:15.418088637 +0000 UTC m=+41.369655693 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cc6bt" (UID: "b0cbd483-407c-4f29-a033-fa0beda9bba7") : secret "insights-runtime-extractor-tls" not found
Jul 29 18:35:14.418141 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:14.418130    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm podName:b0cbd483-407c-4f29-a033-fa0beda9bba7 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:15.418113721 +0000 UTC m=+41.369680775 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cc6bt" (UID: "b0cbd483-407c-4f29-a033-fa0beda9bba7") : configmap "kube-rbac-proxy" not found
Jul 29 18:35:14.836291 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:14.836265    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/0.log"
Jul 29 18:35:14.836772 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:14.836307    2579 generic.go:356] "Generic (PLEG): container finished" podID="272b03c5-141c-4a65-b707-e1d547ceab87" containerID="32abc874ee3a557b1f33ec354e6208b2827d1e3e7b7ed166ba5532eec3484bfd" exitCode=255
Jul 29 18:35:14.836772 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:14.836339    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7" event={"ID":"272b03c5-141c-4a65-b707-e1d547ceab87","Type":"ContainerDied","Data":"32abc874ee3a557b1f33ec354e6208b2827d1e3e7b7ed166ba5532eec3484bfd"}
Jul 29 18:35:14.836772 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:14.836648    2579 scope.go:117] "RemoveContainer" containerID="32abc874ee3a557b1f33ec354e6208b2827d1e3e7b7ed166ba5532eec3484bfd"
Jul 29 18:35:15.287681 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:15.287652    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bd5l5_d1aa370e-70cd-4cbc-9af4-508b7a39b9fc/dns-node-resolver/0.log"
Jul 29 18:35:15.426022 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:15.425979    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:15.426187 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:15.426083    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:15.426187 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:15.426098    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 18:35:15.426187 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:15.426165    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm podName:b0cbd483-407c-4f29-a033-fa0beda9bba7 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:17.426149657 +0000 UTC m=+43.377716717 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cc6bt" (UID: "b0cbd483-407c-4f29-a033-fa0beda9bba7") : configmap "kube-rbac-proxy" not found
Jul 29 18:35:15.426372 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:15.426250    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 18:35:15.426372 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:15.426330    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls podName:b0cbd483-407c-4f29-a033-fa0beda9bba7 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:17.426312551 +0000 UTC m=+43.377879612 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cc6bt" (UID: "b0cbd483-407c-4f29-a033-fa0beda9bba7") : secret "insights-runtime-extractor-tls" not found
Jul 29 18:35:15.840133 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:15.840107    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 18:35:15.840537 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:15.840521    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/0.log"
Jul 29 18:35:15.840592 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:15.840555    2579 generic.go:356] "Generic (PLEG): container finished" podID="272b03c5-141c-4a65-b707-e1d547ceab87" containerID="bb1d5bfe04f0faebde5249ad44634ac14709ef0380a9402e296c1787a2f9844e" exitCode=255
Jul 29 18:35:15.840633 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:15.840606    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7" event={"ID":"272b03c5-141c-4a65-b707-e1d547ceab87","Type":"ContainerDied","Data":"bb1d5bfe04f0faebde5249ad44634ac14709ef0380a9402e296c1787a2f9844e"}
Jul 29 18:35:15.840686 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:15.840673    2579 scope.go:117] "RemoveContainer" containerID="32abc874ee3a557b1f33ec354e6208b2827d1e3e7b7ed166ba5532eec3484bfd"
Jul 29 18:35:15.840890 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:15.840874    2579 scope.go:117] "RemoveContainer" containerID="bb1d5bfe04f0faebde5249ad44634ac14709ef0380a9402e296c1787a2f9844e"
Jul 29 18:35:15.841081 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:15.841062    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-f8wr7_openshift-console-operator(272b03c5-141c-4a65-b707-e1d547ceab87)\"" pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7" podUID="272b03c5-141c-4a65-b707-e1d547ceab87"
Jul 29 18:35:16.086066 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:16.086039    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6vrk4_dcfea240-b8d3-4728-b24f-4fe8268f6d05/node-ca/0.log"
Jul 29 18:35:16.684907 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:16.684871    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6z9cl_0dd1da74-2c12-4022-be1e-3b0879d4c0ba/serve-healthcheck-canary/0.log"
Jul 29 18:35:16.739402 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:16.739352    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gzvjm\" (UID: \"aedf1ecb-e610-4bf3-9688-a969c14dd65b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:16.739402 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:16.739402    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:16.739674 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:16.739421    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:16.739674 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:16.739455    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-xrcrz\" (UID: \"fb928e9d-7a00-471f-ae5e-95524175cb81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"
Jul 29 18:35:16.739674 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:16.739514    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 18:35:16.739674 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:16.739530    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 18:35:16.739674 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:16.739551    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 18:35:16.739674 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:16.739570    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d548586c6-6g7fk: secret "image-registry-tls" not found
Jul 29 18:35:16.739674 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:16.739577    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls podName:fb928e9d-7a00-471f-ae5e-95524175cb81 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:24.739564171 +0000 UTC m=+50.691131226 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls") pod "cluster-samples-operator-6748489656-xrcrz" (UID: "fb928e9d-7a00-471f-ae5e-95524175cb81") : secret "samples-operator-tls" not found
Jul 29 18:35:16.739674 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:16.739592    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls podName:aedf1ecb-e610-4bf3-9688-a969c14dd65b nodeName:}" failed. No retries permitted until 2026-07-29 18:35:24.739584418 +0000 UTC m=+50.691151473 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-gzvjm" (UID: "aedf1ecb-e610-4bf3-9688-a969c14dd65b") : secret "cluster-monitoring-operator-tls" not found
Jul 29 18:35:16.739674 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:16.739632    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle podName:abbf5f2c-044a-4007-9656-54b2b1c454fb nodeName:}" failed. No retries permitted until 2026-07-29 18:35:24.739615229 +0000 UTC m=+50.691182287 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle") pod "router-default-766fd78c95-kbbsx" (UID: "abbf5f2c-044a-4007-9656-54b2b1c454fb") : configmap references non-existent config key: service-ca.crt
Jul 29 18:35:16.739674 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:16.739662    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls podName:cb702a20-d5e5-4070-b422-649738adff18 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:24.7396541 +0000 UTC m=+50.691221159 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls") pod "image-registry-6d548586c6-6g7fk" (UID: "cb702a20-d5e5-4070-b422-649738adff18") : secret "image-registry-tls" not found
Jul 29 18:35:16.740001 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:16.739690    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:16.740001 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:16.739731    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:16.740001 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:16.739801    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 18:35:16.740001 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:16.739832    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 18:35:16.740001 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:16.739857    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs podName:abbf5f2c-044a-4007-9656-54b2b1c454fb nodeName:}" failed. No retries permitted until 2026-07-29 18:35:24.739844922 +0000 UTC m=+50.691412007 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs") pod "router-default-766fd78c95-kbbsx" (UID: "abbf5f2c-044a-4007-9656-54b2b1c454fb") : secret "router-metrics-certs-default" not found
Jul 29 18:35:16.740001 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:16.739873    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls podName:66f3a250-259a-4db3-8a53-bba41080d0cd nodeName:}" failed. No retries permitted until 2026-07-29 18:35:24.739865252 +0000 UTC m=+50.691432307 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls") pod "dns-default-kdnwg" (UID: "66f3a250-259a-4db3-8a53-bba41080d0cd") : secret "dns-default-metrics-tls" not found
Jul 29 18:35:16.844214 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:16.844181    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 18:35:16.844692 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:16.844489    2579 scope.go:117] "RemoveContainer" containerID="bb1d5bfe04f0faebde5249ad44634ac14709ef0380a9402e296c1787a2f9844e"
Jul 29 18:35:16.844692 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:16.844673    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-f8wr7_openshift-console-operator(272b03c5-141c-4a65-b707-e1d547ceab87)\"" pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7" podUID="272b03c5-141c-4a65-b707-e1d547ceab87"
Jul 29 18:35:17.446451 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:17.446410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:17.446755 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:17.446520    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:17.446755 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:17.446553    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 18:35:17.446755 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:17.446622    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm podName:b0cbd483-407c-4f29-a033-fa0beda9bba7 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:21.446605405 +0000 UTC m=+47.398172479 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cc6bt" (UID: "b0cbd483-407c-4f29-a033-fa0beda9bba7") : configmap "kube-rbac-proxy" not found
Jul 29 18:35:17.446755 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:17.446672    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 18:35:17.446755 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:17.446741    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls podName:b0cbd483-407c-4f29-a033-fa0beda9bba7 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:21.446723234 +0000 UTC m=+47.398290294 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cc6bt" (UID: "b0cbd483-407c-4f29-a033-fa0beda9bba7") : secret "insights-runtime-extractor-tls" not found
Jul 29 18:35:19.319358 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:19.319317    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:19.319760 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:19.319372    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:19.319760 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:19.319703    2579 scope.go:117] "RemoveContainer" containerID="bb1d5bfe04f0faebde5249ad44634ac14709ef0380a9402e296c1787a2f9844e"
Jul 29 18:35:19.319857 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:19.319839    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-f8wr7_openshift-console-operator(272b03c5-141c-4a65-b707-e1d547ceab87)\"" pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7" podUID="272b03c5-141c-4a65-b707-e1d547ceab87"
Jul 29 18:35:21.481251 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:21.481205    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:21.481676 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:21.481344    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:21.481676 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:21.481388    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 18:35:21.481676 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:21.481462    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls podName:b0cbd483-407c-4f29-a033-fa0beda9bba7 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:29.481445758 +0000 UTC m=+55.433012830 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cc6bt" (UID: "b0cbd483-407c-4f29-a033-fa0beda9bba7") : secret "insights-runtime-extractor-tls" not found
Jul 29 18:35:21.481676 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:21.481462    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 18:35:21.481676 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:21.481537    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm podName:b0cbd483-407c-4f29-a033-fa0beda9bba7 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:29.481520848 +0000 UTC m=+55.433087920 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cc6bt" (UID: "b0cbd483-407c-4f29-a033-fa0beda9bba7") : configmap "kube-rbac-proxy" not found
Jul 29 18:35:24.810706 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:24.810663    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-xrcrz\" (UID: \"fb928e9d-7a00-471f-ae5e-95524175cb81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"
Jul 29 18:35:24.811157 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:24.810719    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:24.811157 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:24.810749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:24.811157 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:24.810803    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gzvjm\" (UID: \"aedf1ecb-e610-4bf3-9688-a969c14dd65b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:24.811157 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:24.810822    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:24.811157 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:24.810840    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:24.811157 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:24.810820    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 18:35:24.811157 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:24.810875    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 18:35:24.811157 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:24.810905    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 18:35:24.811157 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:24.810868    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 18:35:24.811157 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:24.810945    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls podName:aedf1ecb-e610-4bf3-9688-a969c14dd65b nodeName:}" failed. No retries permitted until 2026-07-29 18:35:40.810925243 +0000 UTC m=+66.762492319 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-gzvjm" (UID: "aedf1ecb-e610-4bf3-9688-a969c14dd65b") : secret "cluster-monitoring-operator-tls" not found
Jul 29 18:35:24.811157 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:24.810935    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 18:35:24.811157 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:24.810963    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d548586c6-6g7fk: secret "image-registry-tls" not found
Jul 29 18:35:24.811157 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:24.810968    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs podName:abbf5f2c-044a-4007-9656-54b2b1c454fb nodeName:}" failed. No retries permitted until 2026-07-29 18:35:40.810953236 +0000 UTC m=+66.762520295 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs") pod "router-default-766fd78c95-kbbsx" (UID: "abbf5f2c-044a-4007-9656-54b2b1c454fb") : secret "router-metrics-certs-default" not found
Jul 29 18:35:24.811157 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:24.810982    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls podName:fb928e9d-7a00-471f-ae5e-95524175cb81 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:40.810975411 +0000 UTC m=+66.762542466 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls") pod "cluster-samples-operator-6748489656-xrcrz" (UID: "fb928e9d-7a00-471f-ae5e-95524175cb81") : secret "samples-operator-tls" not found
Jul 29 18:35:24.811157 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:24.810993    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle podName:abbf5f2c-044a-4007-9656-54b2b1c454fb nodeName:}" failed. No retries permitted until 2026-07-29 18:35:40.810985975 +0000 UTC m=+66.762553030 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle") pod "router-default-766fd78c95-kbbsx" (UID: "abbf5f2c-044a-4007-9656-54b2b1c454fb") : configmap references non-existent config key: service-ca.crt
Jul 29 18:35:24.811157 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:24.811015    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls podName:cb702a20-d5e5-4070-b422-649738adff18 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:40.811001834 +0000 UTC m=+66.762568890 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls") pod "image-registry-6d548586c6-6g7fk" (UID: "cb702a20-d5e5-4070-b422-649738adff18") : secret "image-registry-tls" not found
Jul 29 18:35:24.811705 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:24.811030    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls podName:66f3a250-259a-4db3-8a53-bba41080d0cd nodeName:}" failed. No retries permitted until 2026-07-29 18:35:40.811021907 +0000 UTC m=+66.762588964 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls") pod "dns-default-kdnwg" (UID: "66f3a250-259a-4db3-8a53-bba41080d0cd") : secret "dns-default-metrics-tls" not found
Jul 29 18:35:29.551655 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:29.551595    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:29.552079 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:29.551742    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:29.552079 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:29.551891    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 18:35:29.552079 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:29.551961    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls podName:b0cbd483-407c-4f29-a033-fa0beda9bba7 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:45.551945933 +0000 UTC m=+71.503512987 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cc6bt" (UID: "b0cbd483-407c-4f29-a033-fa0beda9bba7") : secret "insights-runtime-extractor-tls" not found
Jul 29 18:35:29.552195 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:29.552174    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b0cbd483-407c-4f29-a033-fa0beda9bba7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:32.579424 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:32.579392    2579 scope.go:117] "RemoveContainer" containerID="bb1d5bfe04f0faebde5249ad44634ac14709ef0380a9402e296c1787a2f9844e"
Jul 29 18:35:32.883812 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:32.883735    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 18:35:32.883812 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:32.883783    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7" event={"ID":"272b03c5-141c-4a65-b707-e1d547ceab87","Type":"ContainerStarted","Data":"06363e060f63e0454a50f2f387413680da5f3e2b0e6e7afe60b3cb9117e49991"}
Jul 29 18:35:32.884119 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:32.884095    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:32.902582 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:32.902490    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7" podStartSLOduration=21.086497722 podStartE2EDuration="25.902471482s" totalImagesPullingTime="4.81597376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:32.901630052 +0000 UTC m=+58.853197130" watchObservedRunningTime="2026-07-29 18:35:32.902471482 +0000 UTC m=+58.854038568"
Jul 29 18:35:33.260127 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:33.260099    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-f8wr7"
Jul 29 18:35:33.797208 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:33.797176    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-998nt"
Jul 29 18:35:39.230571 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.230495    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret\") pod \"global-pull-secret-syncer-d9d6c\" (UID: \"32e483de-2fdb-429d-bb01-5f7ea6a0e46b\") " pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:35:39.233676 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.233656    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 18:35:39.243367 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.243342    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/32e483de-2fdb-429d-bb01-5f7ea6a0e46b-original-pull-secret\") pod \"global-pull-secret-syncer-d9d6c\" (UID: \"32e483de-2fdb-429d-bb01-5f7ea6a0e46b\") " pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:35:39.331270 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.331235    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs\") pod \"network-metrics-daemon-2vbxt\" (UID: \"ef2dbc84-9591-4b1b-8613-7887422229f2\") " pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:35:39.333983 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.333962    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 18:35:39.344471 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.344436    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2dbc84-9591-4b1b-8613-7887422229f2-metrics-certs\") pod \"network-metrics-daemon-2vbxt\" (UID: \"ef2dbc84-9591-4b1b-8613-7887422229f2\") " pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:35:39.392292 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.392262    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9d6c"
Jul 29 18:35:39.432654 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.432613    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsw6v\" (UniqueName: \"kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v\") pod \"network-check-target-2dp6b\" (UID: \"cbaacddf-394f-43c1-b02b-f85d651ebe28\") " pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:35:39.437061 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.437032    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 18:35:39.446823 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.446791    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 18:35:39.457687 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.457524    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsw6v\" (UniqueName: \"kubernetes.io/projected/cbaacddf-394f-43c1-b02b-f85d651ebe28-kube-api-access-hsw6v\") pod \"network-check-target-2dp6b\" (UID: \"cbaacddf-394f-43c1-b02b-f85d651ebe28\") " pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:35:39.527923 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.527893    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d9d6c"]
Jul 29 18:35:39.531270 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:35:39.531244    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32e483de_2fdb_429d_bb01_5f7ea6a0e46b.slice/crio-e8279495b4b4f6dab9e25fc45bf086ca7403f82ecac5f17e38f970f8dc5a965d WatchSource:0}: Error finding container e8279495b4b4f6dab9e25fc45bf086ca7403f82ecac5f17e38f970f8dc5a965d: Status 404 returned error can't find the container with id e8279495b4b4f6dab9e25fc45bf086ca7403f82ecac5f17e38f970f8dc5a965d
Jul 29 18:35:39.591744 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.591710    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g7g2r\""
Jul 29 18:35:39.599176 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.599149    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2vbxt"
Jul 29 18:35:39.702651 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.702617    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b24qk\""
Jul 29 18:35:39.710842 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.710815    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:35:39.733379 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.733352    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2vbxt"]
Jul 29 18:35:39.746181 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:35:39.746152    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef2dbc84_9591_4b1b_8613_7887422229f2.slice/crio-7f37a9ab9006daa550f92bd6ed16e4f65f9034fc39185708ed67f938267c50b5 WatchSource:0}: Error finding container 7f37a9ab9006daa550f92bd6ed16e4f65f9034fc39185708ed67f938267c50b5: Status 404 returned error can't find the container with id 7f37a9ab9006daa550f92bd6ed16e4f65f9034fc39185708ed67f938267c50b5
Jul 29 18:35:39.858396 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.858365    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2dp6b"]
Jul 29 18:35:39.861291 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:35:39.861259    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbaacddf_394f_43c1_b02b_f85d651ebe28.slice/crio-0824c003a976b36f9382efe3d985d1ecb4c064a7332d37197cdb0b586f571bd7 WatchSource:0}: Error finding container 0824c003a976b36f9382efe3d985d1ecb4c064a7332d37197cdb0b586f571bd7: Status 404 returned error can't find the container with id 0824c003a976b36f9382efe3d985d1ecb4c064a7332d37197cdb0b586f571bd7
Jul 29 18:35:39.904164 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.904123    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2dp6b" event={"ID":"cbaacddf-394f-43c1-b02b-f85d651ebe28","Type":"ContainerStarted","Data":"0824c003a976b36f9382efe3d985d1ecb4c064a7332d37197cdb0b586f571bd7"}
Jul 29 18:35:39.905255 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.905232    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d9d6c" event={"ID":"32e483de-2fdb-429d-bb01-5f7ea6a0e46b","Type":"ContainerStarted","Data":"e8279495b4b4f6dab9e25fc45bf086ca7403f82ecac5f17e38f970f8dc5a965d"}
Jul 29 18:35:39.906339 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:39.906320    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2vbxt" event={"ID":"ef2dbc84-9591-4b1b-8613-7887422229f2","Type":"ContainerStarted","Data":"7f37a9ab9006daa550f92bd6ed16e4f65f9034fc39185708ed67f938267c50b5"}
Jul 29 18:35:40.845241 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:40.845202    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gzvjm\" (UID: \"aedf1ecb-e610-4bf3-9688-a969c14dd65b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:40.845740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:40.845254    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:40.845740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:40.845280    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:40.845740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:40.845320    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-xrcrz\" (UID: \"fb928e9d-7a00-471f-ae5e-95524175cb81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"
Jul 29 18:35:40.845740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:40.845361    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:40.845740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:40.845391    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:40.846819 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:40.846446    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abbf5f2c-044a-4007-9656-54b2b1c454fb-service-ca-bundle\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:40.848555 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:40.848494    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aedf1ecb-e610-4bf3-9688-a969c14dd65b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gzvjm\" (UID: \"aedf1ecb-e610-4bf3-9688-a969c14dd65b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:40.848725 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:40.848704    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66f3a250-259a-4db3-8a53-bba41080d0cd-metrics-tls\") pod \"dns-default-kdnwg\" (UID: \"66f3a250-259a-4db3-8a53-bba41080d0cd\") " pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:40.849740 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:40.849712    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abbf5f2c-044a-4007-9656-54b2b1c454fb-metrics-certs\") pod \"router-default-766fd78c95-kbbsx\" (UID: \"abbf5f2c-044a-4007-9656-54b2b1c454fb\") " pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:40.850191 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:40.850147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb928e9d-7a00-471f-ae5e-95524175cb81-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-xrcrz\" (UID: \"fb928e9d-7a00-471f-ae5e-95524175cb81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"
Jul 29 18:35:40.850288 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:40.850192    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls\") pod \"image-registry-6d548586c6-6g7fk\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") " pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:41.084830 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.084787    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v2s6m\""
Jul 29 18:35:41.090647 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.090387    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-q6xm9\""
Jul 29 18:35:41.091844 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.091816    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:41.098445 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.098329    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"
Jul 29 18:35:41.100855 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.100619    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rgfqx\""
Jul 29 18:35:41.107729 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.107634    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-xvlk7\""
Jul 29 18:35:41.108783 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.108704    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:41.115467 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.115427    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"
Jul 29 18:35:41.137861 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.137620    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hgmvm\""
Jul 29 18:35:41.145548 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.145515    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:41.456563 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.455569    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz"]
Jul 29 18:35:41.480165 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.480120    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-766fd78c95-kbbsx"]
Jul 29 18:35:41.594146 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:35:41.593199    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabbf5f2c_044a_4007_9656_54b2b1c454fb.slice/crio-d7b2c97fa8412de22959e80c7053d3bb4ccc0dc57a7e3378ba4c78db8eea9734 WatchSource:0}: Error finding container d7b2c97fa8412de22959e80c7053d3bb4ccc0dc57a7e3378ba4c78db8eea9734: Status 404 returned error can't find the container with id d7b2c97fa8412de22959e80c7053d3bb4ccc0dc57a7e3378ba4c78db8eea9734
Jul 29 18:35:41.711422 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.711381    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kdnwg"]
Jul 29 18:35:41.716478 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:35:41.715260    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66f3a250_259a_4db3_8a53_bba41080d0cd.slice/crio-637aafbf481b32c6cd48da9a784c15fdddd108975498e24b348df1076e3346fa WatchSource:0}: Error finding container 637aafbf481b32c6cd48da9a784c15fdddd108975498e24b348df1076e3346fa: Status 404 returned error can't find the container with id 637aafbf481b32c6cd48da9a784c15fdddd108975498e24b348df1076e3346fa
Jul 29 18:35:41.717631 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.717600    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm"]
Jul 29 18:35:41.721524 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:35:41.721460    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaedf1ecb_e610_4bf3_9688_a969c14dd65b.slice/crio-807cb7b00dc045ac4c8fa1fdab042c33a75c70b8b8c8098de1297f6d8016d9a9 WatchSource:0}: Error finding container 807cb7b00dc045ac4c8fa1fdab042c33a75c70b8b8c8098de1297f6d8016d9a9: Status 404 returned error can't find the container with id 807cb7b00dc045ac4c8fa1fdab042c33a75c70b8b8c8098de1297f6d8016d9a9
Jul 29 18:35:41.724854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.724767    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d548586c6-6g7fk"]
Jul 29 18:35:41.728498 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:35:41.728465    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb702a20_d5e5_4070_b422_649738adff18.slice/crio-e1557654f1c58d20a73100b647efe885e9d50ce3eef8b65f95acaa4da251d2f9 WatchSource:0}: Error finding container e1557654f1c58d20a73100b647efe885e9d50ce3eef8b65f95acaa4da251d2f9: Status 404 returned error can't find the container with id e1557654f1c58d20a73100b647efe885e9d50ce3eef8b65f95acaa4da251d2f9
Jul 29 18:35:41.916286 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.915904    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-766fd78c95-kbbsx" event={"ID":"abbf5f2c-044a-4007-9656-54b2b1c454fb","Type":"ContainerStarted","Data":"bd01b02548feaa4ef9452844ca565752733e2207b7ad945729decab625280e3b"}
Jul 29 18:35:41.916286 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.916010    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-766fd78c95-kbbsx" event={"ID":"abbf5f2c-044a-4007-9656-54b2b1c454fb","Type":"ContainerStarted","Data":"d7b2c97fa8412de22959e80c7053d3bb4ccc0dc57a7e3378ba4c78db8eea9734"}
Jul 29 18:35:41.918046 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.918016    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kdnwg" event={"ID":"66f3a250-259a-4db3-8a53-bba41080d0cd","Type":"ContainerStarted","Data":"637aafbf481b32c6cd48da9a784c15fdddd108975498e24b348df1076e3346fa"}
Jul 29 18:35:41.919930 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.919897    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2vbxt" event={"ID":"ef2dbc84-9591-4b1b-8613-7887422229f2","Type":"ContainerStarted","Data":"534dbe23b38d391b0af5862eb3456b75e0a7f09b7baaea535b98e9ac8f99a283"}
Jul 29 18:35:41.921672 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.921632    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm" event={"ID":"aedf1ecb-e610-4bf3-9688-a969c14dd65b","Type":"ContainerStarted","Data":"807cb7b00dc045ac4c8fa1fdab042c33a75c70b8b8c8098de1297f6d8016d9a9"}
Jul 29 18:35:41.923386 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.923355    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz" event={"ID":"fb928e9d-7a00-471f-ae5e-95524175cb81","Type":"ContainerStarted","Data":"9a987d17879199e142d0822dd7d61ff696381782c92d56b9344c29d0130fd668"}
Jul 29 18:35:41.925592 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.925565    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d548586c6-6g7fk" event={"ID":"cb702a20-d5e5-4070-b422-649738adff18","Type":"ContainerStarted","Data":"b280b9fbd822543e07155cf2338e31719d98c5a3376ece01524aa9bcdb43f01e"}
Jul 29 18:35:41.925686 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.925613    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d548586c6-6g7fk" event={"ID":"cb702a20-d5e5-4070-b422-649738adff18","Type":"ContainerStarted","Data":"e1557654f1c58d20a73100b647efe885e9d50ce3eef8b65f95acaa4da251d2f9"}
Jul 29 18:35:41.925797 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.925779    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:35:41.937682 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.937635    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-766fd78c95-kbbsx" podStartSLOduration=35.937617487 podStartE2EDuration="35.937617487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:41.936803771 +0000 UTC m=+67.888370849" watchObservedRunningTime="2026-07-29 18:35:41.937617487 +0000 UTC m=+67.889184564"
Jul 29 18:35:41.962024 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:41.960955    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d548586c6-6g7fk" podStartSLOduration=67.960934287 podStartE2EDuration="1m7.960934287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:34:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:41.960687878 +0000 UTC m=+67.912254953" watchObservedRunningTime="2026-07-29 18:35:41.960934287 +0000 UTC m=+67.912501366"
Jul 29 18:35:42.109297 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:42.109251    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:42.112725 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:42.112688    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:42.933157 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:42.933074    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2vbxt" event={"ID":"ef2dbc84-9591-4b1b-8613-7887422229f2","Type":"ContainerStarted","Data":"e122229e0da9ab6afaf8b34241d320e1b47ccefb6a1c797a32cd3ce39b0752b3"}
Jul 29 18:35:42.933791 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:42.933769    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:42.934876 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:42.934848    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-766fd78c95-kbbsx"
Jul 29 18:35:42.952518 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:42.952449    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2vbxt" podStartSLOduration=67.08065328 podStartE2EDuration="1m8.952425664s" totalImagesPullingTime="1.871772384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:42.951638975 +0000 UTC m=+68.903206064" watchObservedRunningTime="2026-07-29 18:35:42.952425664 +0000 UTC m=+68.903992741"
Jul 29 18:35:44.403180 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.403022    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-ptp2l"]
Jul 29 18:35:44.427016 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.426975    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-878ht"]
Jul 29 18:35:44.427213 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.427083    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-ptp2l"
Jul 29 18:35:44.430820 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.430794    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tsvxl\""
Jul 29 18:35:44.430993 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.430853    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 29 18:35:44.430993 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.430885    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 29 18:35:44.450755 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.450717    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-ptp2l"]
Jul 29 18:35:44.450755 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.450755    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-878ht"]
Jul 29 18:35:44.451001 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.450862    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-878ht"
Jul 29 18:35:44.453496 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.453467    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 18:35:44.453663 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.453620    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-n422v\""
Jul 29 18:35:44.453870 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.453805    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 18:35:44.480073 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.480040    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7c6804ba-7543-4df8-8ca8-d5b463078329-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-ptp2l\" (UID: \"7c6804ba-7543-4df8-8ca8-d5b463078329\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ptp2l"
Jul 29 18:35:44.480269 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.480247    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c6804ba-7543-4df8-8ca8-d5b463078329-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ptp2l\" (UID: \"7c6804ba-7543-4df8-8ca8-d5b463078329\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ptp2l"
Jul 29 18:35:44.580756 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.580659    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c6804ba-7543-4df8-8ca8-d5b463078329-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ptp2l\" (UID: \"7c6804ba-7543-4df8-8ca8-d5b463078329\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ptp2l"
Jul 29 18:35:44.580756 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.580719    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7c6804ba-7543-4df8-8ca8-d5b463078329-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-ptp2l\" (UID: \"7c6804ba-7543-4df8-8ca8-d5b463078329\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ptp2l"
Jul 29 18:35:44.580983 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.580758    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px6wf\" (UniqueName: \"kubernetes.io/projected/9eeaeb36-0549-4b09-9663-7d77720fd5b4-kube-api-access-px6wf\") pod \"downloads-7c97dc4555-878ht\" (UID: \"9eeaeb36-0549-4b09-9663-7d77720fd5b4\") " pod="openshift-console/downloads-7c97dc4555-878ht"
Jul 29 18:35:44.581531 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.581471    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7c6804ba-7543-4df8-8ca8-d5b463078329-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-ptp2l\" (UID: \"7c6804ba-7543-4df8-8ca8-d5b463078329\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ptp2l"
Jul 29 18:35:44.583792 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.583752    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c6804ba-7543-4df8-8ca8-d5b463078329-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-ptp2l\" (UID: \"7c6804ba-7543-4df8-8ca8-d5b463078329\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-ptp2l"
Jul 29 18:35:44.681688 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.681649    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px6wf\" (UniqueName: \"kubernetes.io/projected/9eeaeb36-0549-4b09-9663-7d77720fd5b4-kube-api-access-px6wf\") pod \"downloads-7c97dc4555-878ht\" (UID: \"9eeaeb36-0549-4b09-9663-7d77720fd5b4\") " pod="openshift-console/downloads-7c97dc4555-878ht"
Jul 29 18:35:44.691573 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.691539    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px6wf\" (UniqueName: \"kubernetes.io/projected/9eeaeb36-0549-4b09-9663-7d77720fd5b4-kube-api-access-px6wf\") pod \"downloads-7c97dc4555-878ht\" (UID: \"9eeaeb36-0549-4b09-9663-7d77720fd5b4\") " pod="openshift-console/downloads-7c97dc4555-878ht"
Jul 29 18:35:44.738420 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.738376    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-ptp2l"
Jul 29 18:35:44.762879 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:44.762842    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-878ht"
Jul 29 18:35:45.589595 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:45.589557    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:45.592522 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:45.592478    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0cbd483-407c-4f29-a033-fa0beda9bba7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cc6bt\" (UID: \"b0cbd483-407c-4f29-a033-fa0beda9bba7\") " pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:45.834693 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:45.834645    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8px9m\""
Jul 29 18:35:45.842930 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:45.842829    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cc6bt"
Jul 29 18:35:46.697731 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:46.697498    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-ptp2l"]
Jul 29 18:35:46.740515 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:46.740419    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-878ht"]
Jul 29 18:35:46.747147 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:35:46.747104    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eeaeb36_0549_4b09_9663_7d77720fd5b4.slice/crio-ca8b5ee5b71e7dc1c1dd4d9743e5edb576d1cb488eafc7cfab0321e1ead3c6c1 WatchSource:0}: Error finding container ca8b5ee5b71e7dc1c1dd4d9743e5edb576d1cb488eafc7cfab0321e1ead3c6c1: Status 404 returned error can't find the container with id ca8b5ee5b71e7dc1c1dd4d9743e5edb576d1cb488eafc7cfab0321e1ead3c6c1
Jul 29 18:35:46.771530 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:46.771101    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cc6bt"]
Jul 29 18:35:46.954549 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:46.954498    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm" event={"ID":"aedf1ecb-e610-4bf3-9688-a969c14dd65b","Type":"ContainerStarted","Data":"906eff99b4b71fff5ecaccd65196c84e5f23bb89353902d890dad79947964d60"}
Jul 29 18:35:46.956421 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:46.956393    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz" event={"ID":"fb928e9d-7a00-471f-ae5e-95524175cb81","Type":"ContainerStarted","Data":"e208ed83502d8dbe68aeec8bd85ec2e56614405476d12c010dbb4eda40fd4bbb"}
Jul 29 18:35:46.956566 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:46.956450    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz" event={"ID":"fb928e9d-7a00-471f-ae5e-95524175cb81","Type":"ContainerStarted","Data":"d1a53587c7f1d5ccdae317612a35a8212a447e43ebb9ef9346acde4664f65033"}
Jul 29 18:35:46.957751 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:46.957717    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-ptp2l" event={"ID":"7c6804ba-7543-4df8-8ca8-d5b463078329","Type":"ContainerStarted","Data":"5d336989b07c9c8548ff2df5dabf097ffad4c6845fabae20b8a0a57425065284"}
Jul 29 18:35:46.959370 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:46.959344    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2dp6b" event={"ID":"cbaacddf-394f-43c1-b02b-f85d651ebe28","Type":"ContainerStarted","Data":"6495da8f925d4e7b86f9c63cfe3f349780b8605ce5693b6d3538e04f22271ac8"}
Jul 29 18:35:46.959479 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:46.959405    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:35:46.960876 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:46.960818    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d9d6c" event={"ID":"32e483de-2fdb-429d-bb01-5f7ea6a0e46b","Type":"ContainerStarted","Data":"2a6ab95aacfe7a620a5dfaee7c74eacec6ded724022e511d67432f82abfb0386"}
Jul 29 18:35:46.963476 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:46.963455    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kdnwg" event={"ID":"66f3a250-259a-4db3-8a53-bba41080d0cd","Type":"ContainerStarted","Data":"67bccc9d829344530fd7cd97d44d94c4282f7903a954d0f097d9d4d730ebebbe"}
Jul 29 18:35:46.963601 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:46.963493    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kdnwg" event={"ID":"66f3a250-259a-4db3-8a53-bba41080d0cd","Type":"ContainerStarted","Data":"05e78c461bafd164b909f3fceafc9d591ec5cbd062b3a019c74c0aeff53607d2"}
Jul 29 18:35:46.963601 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:46.963578    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:46.964837 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:46.964816    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-878ht" event={"ID":"9eeaeb36-0549-4b09-9663-7d77720fd5b4","Type":"ContainerStarted","Data":"ca8b5ee5b71e7dc1c1dd4d9743e5edb576d1cb488eafc7cfab0321e1ead3c6c1"}
Jul 29 18:35:46.966041 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:46.966021    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cc6bt" event={"ID":"b0cbd483-407c-4f29-a033-fa0beda9bba7","Type":"ContainerStarted","Data":"a8bfed1f4004c00d0ce3e9946f4db96e03c4574c700f3a3b5711d917d5f1500c"}
Jul 29 18:35:46.978782 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:46.978677    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gzvjm" podStartSLOduration=36.229415168 podStartE2EDuration="40.978657043s" totalImagesPullingTime="4.749241875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:46.977498822 +0000 UTC m=+72.929065900" watchObservedRunningTime="2026-07-29 18:35:46.978657043 +0000 UTC m=+72.930224119"
Jul 29 18:35:47.007162 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:47.007103    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-d9d6c" podStartSLOduration=66.064709283 podStartE2EDuration="1m13.007089938s" totalImagesPullingTime="6.942380655s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 18:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:47.006284417 +0000 UTC m=+72.957851496" watchObservedRunningTime="2026-07-29 18:35:47.007089938 +0000 UTC m=+72.958656992"
Jul 29 18:35:47.032830 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:47.032769    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-xrcrz" podStartSLOduration=36.234209019 podStartE2EDuration="41.032745274s" totalImagesPullingTime="4.798536255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:47.032415875 +0000 UTC m=+72.983982954" watchObservedRunningTime="2026-07-29 18:35:47.032745274 +0000 UTC m=+72.984312351"
Jul 29 18:35:47.052147 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:47.052096    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2dp6b" podStartSLOduration=66.44425369 podStartE2EDuration="1m13.052081454s" totalImagesPullingTime="6.607827764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:47.050631937 +0000 UTC m=+73.002199013" watchObservedRunningTime="2026-07-29 18:35:47.052081454 +0000 UTC m=+73.003648531"
Jul 29 18:35:47.074121 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:47.074069    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-kdnwg" podStartSLOduration=34.281509626 podStartE2EDuration="39.074053197s" totalImagesPullingTime="4.792543571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:47.073435973 +0000 UTC m=+73.025003051" watchObservedRunningTime="2026-07-29 18:35:47.074053197 +0000 UTC m=+73.025620273"
Jul 29 18:35:47.265708 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:47.265672    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hf46b"]
Jul 29 18:35:47.268371 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:47.268339    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hf46b"
Jul 29 18:35:47.272328 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:47.272300    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-h6srb\""
Jul 29 18:35:47.272724 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:47.272704    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 29 18:35:47.284252 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:47.284166    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hf46b"]
Jul 29 18:35:47.414073 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:47.414028    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1bd334fd-5fba-4ee9-a95f-ed7957bec060-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-hf46b\" (UID: \"1bd334fd-5fba-4ee9-a95f-ed7957bec060\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hf46b"
Jul 29 18:35:47.514780 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:47.514741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1bd334fd-5fba-4ee9-a95f-ed7957bec060-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-hf46b\" (UID: \"1bd334fd-5fba-4ee9-a95f-ed7957bec060\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hf46b"
Jul 29 18:35:47.519315 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:47.519169    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1bd334fd-5fba-4ee9-a95f-ed7957bec060-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-hf46b\" (UID: \"1bd334fd-5fba-4ee9-a95f-ed7957bec060\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hf46b"
Jul 29 18:35:47.586809 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:47.586495    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hf46b"
Jul 29 18:35:47.801349 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:47.800934    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hf46b"]
Jul 29 18:35:47.805337 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:35:47.805299    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bd334fd_5fba_4ee9_a95f_ed7957bec060.slice/crio-a1907041c2673da3e6358037d9e80535235cebc57652d01ed72488a2ef9b8aa2 WatchSource:0}: Error finding container a1907041c2673da3e6358037d9e80535235cebc57652d01ed72488a2ef9b8aa2: Status 404 returned error can't find the container with id a1907041c2673da3e6358037d9e80535235cebc57652d01ed72488a2ef9b8aa2
Jul 29 18:35:47.976240 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:47.975583    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hf46b" event={"ID":"1bd334fd-5fba-4ee9-a95f-ed7957bec060","Type":"ContainerStarted","Data":"a1907041c2673da3e6358037d9e80535235cebc57652d01ed72488a2ef9b8aa2"}
Jul 29 18:35:48.982755 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:48.982710    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-ptp2l" event={"ID":"7c6804ba-7543-4df8-8ca8-d5b463078329","Type":"ContainerStarted","Data":"ba27abbc15fa6e03b22d055d56fcd02689f6404783aae8f1ef326bf79b4030b2"}
Jul 29 18:35:49.002169 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:49.002047    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-ptp2l" podStartSLOduration=3.33084511 podStartE2EDuration="5.002023452s" totalImagesPullingTime="1.671178342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:49.000835317 +0000 UTC m=+74.952402389" watchObservedRunningTime="2026-07-29 18:35:49.002023452 +0000 UTC m=+74.953590531"
Jul 29 18:35:49.989214 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:49.989050    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hf46b" event={"ID":"1bd334fd-5fba-4ee9-a95f-ed7957bec060","Type":"ContainerStarted","Data":"71214d10bc23739d428f9bd0c1ea1eefb91a2618d4980c04dc7f116418f1b908"}
Jul 29 18:35:49.989747 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:49.989356    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hf46b"
Jul 29 18:35:49.994921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:49.994864    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hf46b"
Jul 29 18:35:50.006936 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.006874    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-hf46b" podStartSLOduration=1.6308229349999999 podStartE2EDuration="3.006849787s" totalImagesPullingTime="1.376026852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:50.006074364 +0000 UTC m=+75.957641441" watchObservedRunningTime="2026-07-29 18:35:50.006849787 +0000 UTC m=+75.958416844"
Jul 29 18:35:50.312854 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.312821    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"]
Jul 29 18:35:50.321894 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.321859    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"
Jul 29 18:35:50.324938 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.324743    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 29 18:35:50.324938 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.324743    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 29 18:35:50.326006 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.325948    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-klhhb\""
Jul 29 18:35:50.326245 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.326224    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 18:35:50.329175 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.328679    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"]
Jul 29 18:35:50.444892 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.444849    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1de2e57e-5f4f-4db4-9be9-bfb561e48e82-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-ndnmt\" (UID: \"1de2e57e-5f4f-4db4-9be9-bfb561e48e82\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"
Jul 29 18:35:50.445091 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.444964    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2wqv\" (UniqueName: \"kubernetes.io/projected/1de2e57e-5f4f-4db4-9be9-bfb561e48e82-kube-api-access-z2wqv\") pod \"prometheus-operator-84d69f794d-ndnmt\" (UID: \"1de2e57e-5f4f-4db4-9be9-bfb561e48e82\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"
Jul 29 18:35:50.445160 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.445085    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1de2e57e-5f4f-4db4-9be9-bfb561e48e82-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-ndnmt\" (UID: \"1de2e57e-5f4f-4db4-9be9-bfb561e48e82\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"
Jul 29 18:35:50.445160 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.445136    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1de2e57e-5f4f-4db4-9be9-bfb561e48e82-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-ndnmt\" (UID: \"1de2e57e-5f4f-4db4-9be9-bfb561e48e82\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"
Jul 29 18:35:50.546595 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.546551    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2wqv\" (UniqueName: \"kubernetes.io/projected/1de2e57e-5f4f-4db4-9be9-bfb561e48e82-kube-api-access-z2wqv\") pod \"prometheus-operator-84d69f794d-ndnmt\" (UID: \"1de2e57e-5f4f-4db4-9be9-bfb561e48e82\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"
Jul 29 18:35:50.546782 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.546660    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1de2e57e-5f4f-4db4-9be9-bfb561e48e82-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-ndnmt\" (UID: \"1de2e57e-5f4f-4db4-9be9-bfb561e48e82\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"
Jul 29 18:35:50.546782 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.546704    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1de2e57e-5f4f-4db4-9be9-bfb561e48e82-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-ndnmt\" (UID: \"1de2e57e-5f4f-4db4-9be9-bfb561e48e82\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"
Jul 29 18:35:50.546782 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.546735    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1de2e57e-5f4f-4db4-9be9-bfb561e48e82-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-ndnmt\" (UID: \"1de2e57e-5f4f-4db4-9be9-bfb561e48e82\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"
Jul 29 18:35:50.547801 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.547722    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1de2e57e-5f4f-4db4-9be9-bfb561e48e82-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-ndnmt\" (UID: \"1de2e57e-5f4f-4db4-9be9-bfb561e48e82\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"
Jul 29 18:35:50.550356 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.550325    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1de2e57e-5f4f-4db4-9be9-bfb561e48e82-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-ndnmt\" (UID: \"1de2e57e-5f4f-4db4-9be9-bfb561e48e82\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"
Jul 29 18:35:50.550565 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.550537    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1de2e57e-5f4f-4db4-9be9-bfb561e48e82-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-ndnmt\" (UID: \"1de2e57e-5f4f-4db4-9be9-bfb561e48e82\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"
Jul 29 18:35:50.555791 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.555733    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2wqv\" (UniqueName: \"kubernetes.io/projected/1de2e57e-5f4f-4db4-9be9-bfb561e48e82-kube-api-access-z2wqv\") pod \"prometheus-operator-84d69f794d-ndnmt\" (UID: \"1de2e57e-5f4f-4db4-9be9-bfb561e48e82\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"
Jul 29 18:35:50.635797 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:50.635701    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"
Jul 29 18:35:53.200461 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:53.200419    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-ndnmt"]
Jul 29 18:35:53.205820 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:35:53.205784    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1de2e57e_5f4f_4db4_9be9_bfb561e48e82.slice/crio-8a04027c4301a966b5c88abd37207bd1c9f8a3a9e48f0b8f1b221b7bd4352e75 WatchSource:0}: Error finding container 8a04027c4301a966b5c88abd37207bd1c9f8a3a9e48f0b8f1b221b7bd4352e75: Status 404 returned error can't find the container with id 8a04027c4301a966b5c88abd37207bd1c9f8a3a9e48f0b8f1b221b7bd4352e75
Jul 29 18:35:54.007978 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:54.007933    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cc6bt" event={"ID":"b0cbd483-407c-4f29-a033-fa0beda9bba7","Type":"ContainerStarted","Data":"a645120ae7d6083e4910a4cc2eebb474f54a10707cd912479a8c5ac2bb97d732"}
Jul 29 18:35:54.009410 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:54.009350    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt" event={"ID":"1de2e57e-5f4f-4db4-9be9-bfb561e48e82","Type":"ContainerStarted","Data":"8a04027c4301a966b5c88abd37207bd1c9f8a3a9e48f0b8f1b221b7bd4352e75"}
Jul 29 18:35:55.017521 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:55.017476    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt" event={"ID":"1de2e57e-5f4f-4db4-9be9-bfb561e48e82","Type":"ContainerStarted","Data":"42ce800f61e79792c433e884ffb7ed270a7c948cdc83b4fdbbbe69c9090664b6"}
Jul 29 18:35:55.018016 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:55.017565    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt" event={"ID":"1de2e57e-5f4f-4db4-9be9-bfb561e48e82","Type":"ContainerStarted","Data":"2bcf3dd00d0e19c8cb6f60e648d2b037537335b4546b2424a5eb30cf782aeb6c"}
Jul 29 18:35:55.038248 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:55.037890    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-ndnmt" podStartSLOduration=3.772778959 podStartE2EDuration="5.037860444s" totalImagesPullingTime="1.265081485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:35:55.034137209 +0000 UTC m=+80.985704309" watchObservedRunningTime="2026-07-29 18:35:55.037860444 +0000 UTC m=+80.989427546"
Jul 29 18:35:56.688636 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.688497    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"]
Jul 29 18:35:56.701620 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.701585    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"
Jul 29 18:35:56.706851 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.706428    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 29 18:35:56.706851 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.706645    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 29 18:35:56.706851 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.706734    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-59qxw\""
Jul 29 18:35:56.709475 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.709440    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"]
Jul 29 18:35:56.734485 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.734451    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fq9lf"]
Jul 29 18:35:56.738580 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.738553    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:56.741536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.741484    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 18:35:56.741536 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.741500    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 18:35:56.741785 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.741759    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 18:35:56.741926 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.741790    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2fkkq\""
Jul 29 18:35:56.798837 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.798790    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj4nv\" (UniqueName: \"kubernetes.io/projected/5e4c363b-bd03-40d1-b561-7be506768d71-kube-api-access-dj4nv\") pod \"openshift-state-metrics-b8f49f5c6-6p9hg\" (UID: \"5e4c363b-bd03-40d1-b561-7be506768d71\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"
Jul 29 18:35:56.799011 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.798857    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e4c363b-bd03-40d1-b561-7be506768d71-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-6p9hg\" (UID: \"5e4c363b-bd03-40d1-b561-7be506768d71\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"
Jul 29 18:35:56.799011 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.798878    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e4c363b-bd03-40d1-b561-7be506768d71-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-6p9hg\" (UID: \"5e4c363b-bd03-40d1-b561-7be506768d71\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"
Jul 29 18:35:56.799011 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.798915    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e4c363b-bd03-40d1-b561-7be506768d71-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6p9hg\" (UID: \"5e4c363b-bd03-40d1-b561-7be506768d71\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"
Jul 29 18:35:56.900262 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.900174    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e4c363b-bd03-40d1-b561-7be506768d71-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-6p9hg\" (UID: \"5e4c363b-bd03-40d1-b561-7be506768d71\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"
Jul 29 18:35:56.900262 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.900233    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e4c363b-bd03-40d1-b561-7be506768d71-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-6p9hg\" (UID: \"5e4c363b-bd03-40d1-b561-7be506768d71\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"
Jul 29 18:35:56.900483 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.900274    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-root\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:56.900483 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.900298    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-node-exporter-textfile\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:56.900483 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.900324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-metrics-client-ca\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:56.900688 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.900365    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e4c363b-bd03-40d1-b561-7be506768d71-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6p9hg\" (UID: \"5e4c363b-bd03-40d1-b561-7be506768d71\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"
Jul 29 18:35:56.900688 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.900555    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-node-exporter-tls\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:56.900688 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.900587    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gtqw\" (UniqueName: \"kubernetes.io/projected/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-kube-api-access-7gtqw\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:56.900688 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.900643    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-sys\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:56.900688 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.900671    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-node-exporter-accelerators-collector-config\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:56.900939 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.900703    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-node-exporter-wtmp\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:56.900939 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.900737    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:56.900939 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.900772    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj4nv\" (UniqueName: \"kubernetes.io/projected/5e4c363b-bd03-40d1-b561-7be506768d71-kube-api-access-dj4nv\") pod \"openshift-state-metrics-b8f49f5c6-6p9hg\" (UID: \"5e4c363b-bd03-40d1-b561-7be506768d71\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"
Jul 29 18:35:56.900939 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:56.900819    2579 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 29 18:35:56.900939 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:35:56.900886    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e4c363b-bd03-40d1-b561-7be506768d71-openshift-state-metrics-tls podName:5e4c363b-bd03-40d1-b561-7be506768d71 nodeName:}" failed. No retries permitted until 2026-07-29 18:35:57.400869438 +0000 UTC m=+83.352436494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5e4c363b-bd03-40d1-b561-7be506768d71-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-6p9hg" (UID: "5e4c363b-bd03-40d1-b561-7be506768d71") : secret "openshift-state-metrics-tls" not found
Jul 29 18:35:56.901184 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.901050    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e4c363b-bd03-40d1-b561-7be506768d71-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-6p9hg\" (UID: \"5e4c363b-bd03-40d1-b561-7be506768d71\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"
Jul 29 18:35:56.903248 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.903191    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e4c363b-bd03-40d1-b561-7be506768d71-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-6p9hg\" (UID: \"5e4c363b-bd03-40d1-b561-7be506768d71\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"
Jul 29 18:35:56.913657 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.913622    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj4nv\" (UniqueName: \"kubernetes.io/projected/5e4c363b-bd03-40d1-b561-7be506768d71-kube-api-access-dj4nv\") pod \"openshift-state-metrics-b8f49f5c6-6p9hg\" (UID: \"5e4c363b-bd03-40d1-b561-7be506768d71\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"
Jul 29 18:35:56.978887 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:56.978793    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kdnwg"
Jul 29 18:35:57.002134 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.002094    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-root\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.002134 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.002140    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-node-exporter-textfile\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.002397 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.002177    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-metrics-client-ca\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.002397 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.002184    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-root\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.002397 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.002234    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-node-exporter-tls\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.002397 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.002261    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gtqw\" (UniqueName: \"kubernetes.io/projected/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-kube-api-access-7gtqw\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.002397 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.002310    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-sys\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.002397 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.002335    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-node-exporter-accelerators-collector-config\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.002397 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.002365    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-node-exporter-wtmp\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.002397 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.002397    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.002804 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.002574    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-node-exporter-textfile\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.002804 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.002640    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-sys\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.002804 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.002696    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-node-exporter-wtmp\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.003369 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.003321    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-node-exporter-accelerators-collector-config\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.003496 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.003409    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-metrics-client-ca\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.005783 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.005732    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.007404 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.007351    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-node-exporter-tls\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.015610 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.015539    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gtqw\" (UniqueName: \"kubernetes.io/projected/53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d-kube-api-access-7gtqw\") pod \"node-exporter-fq9lf\" (UID: \"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d\") " pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.051895 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.051853    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fq9lf"
Jul 29 18:35:57.406905 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.406864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e4c363b-bd03-40d1-b561-7be506768d71-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6p9hg\" (UID: \"5e4c363b-bd03-40d1-b561-7be506768d71\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"
Jul 29 18:35:57.409653 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.409594    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e4c363b-bd03-40d1-b561-7be506768d71-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6p9hg\" (UID: \"5e4c363b-bd03-40d1-b561-7be506768d71\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"
Jul 29 18:35:57.620211 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:57.620170    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"
Jul 29 18:35:58.779465 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.779426    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"]
Jul 29 18:35:58.783094 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.783065    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:58.786181 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.786118    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 29 18:35:58.786395 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.786371    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 29 18:35:58.786543 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.786374    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 29 18:35:58.787298 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.787272    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 29 18:35:58.787404 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.787296    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 29 18:35:58.787404 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.787321    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-sm97p\""
Jul 29 18:35:58.787404 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.787280    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fucaeait1r49r\""
Jul 29 18:35:58.793345 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.793269    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"]
Jul 29 18:35:58.919862 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.919822    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:58.920041 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.919878    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:58.920041 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.919902    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-thanos-querier-tls\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:58.920041 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.919927    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-metrics-client-ca\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:58.920186 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.920035    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:58.920186 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.920076    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npkdc\" (UniqueName: \"kubernetes.io/projected/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-kube-api-access-npkdc\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:58.920186 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.920100    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:58.920186 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:58.920156    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-grpc-tls\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.020847 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.020809    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-metrics-client-ca\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.021043 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.020875    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.021043 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.020905    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npkdc\" (UniqueName: \"kubernetes.io/projected/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-kube-api-access-npkdc\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.021043 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.020939    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.021043 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.020960    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-grpc-tls\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.021043 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.021027    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.021300 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.021056    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.021300 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.021080    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-thanos-querier-tls\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.021785 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.021732    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-metrics-client-ca\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.024279 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.024242    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.024521 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.024482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-grpc-tls\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.024707 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.024680    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-thanos-querier-tls\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.024839 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.024824    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.024887 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.024495    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.024887 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.024819    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.030111 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.030038    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npkdc\" (UniqueName: \"kubernetes.io/projected/0cf3e6f3-df70-4246-bdae-e2c16ba98c34-kube-api-access-npkdc\") pod \"thanos-querier-56dbd69f55-xnzw2\" (UID: \"0cf3e6f3-df70-4246-bdae-e2c16ba98c34\") " pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:35:59.096136 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:35:59.096097    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:36:01.191435 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.191394    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-86969d99b9-xcn44"]
Jul 29 18:36:01.200442 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.200406    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.203382 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.203349    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 18:36:01.204007 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.203979    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 29 18:36:01.204131 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.204096    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 29 18:36:01.204673 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.204652    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-jwmj8\""
Jul 29 18:36:01.204886 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.204866    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-chhtte6b35bq7\""
Jul 29 18:36:01.205169 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.205138    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86969d99b9-xcn44"]
Jul 29 18:36:01.205409 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.205389    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 29 18:36:01.343259 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.343213    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4g4f\" (UniqueName: \"kubernetes.io/projected/4ea7a10f-8208-4922-998e-0e38f321dbce-kube-api-access-v4g4f\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.343450 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.343310    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4ea7a10f-8208-4922-998e-0e38f321dbce-secret-metrics-server-client-certs\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.343450 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.343360    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4ea7a10f-8208-4922-998e-0e38f321dbce-secret-metrics-server-tls\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.343450 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.343410    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ea7a10f-8208-4922-998e-0e38f321dbce-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.343450 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.343441    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4ea7a10f-8208-4922-998e-0e38f321dbce-audit-log\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.343679 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.343472    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ea7a10f-8208-4922-998e-0e38f321dbce-client-ca-bundle\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.343679 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.343499    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4ea7a10f-8208-4922-998e-0e38f321dbce-metrics-server-audit-profiles\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.444559 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.444455    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ea7a10f-8208-4922-998e-0e38f321dbce-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.444559 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.444535    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4ea7a10f-8208-4922-998e-0e38f321dbce-audit-log\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.444771 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.444564    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ea7a10f-8208-4922-998e-0e38f321dbce-client-ca-bundle\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.444771 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.444594    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4ea7a10f-8208-4922-998e-0e38f321dbce-metrics-server-audit-profiles\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.444771 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.444674    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4g4f\" (UniqueName: \"kubernetes.io/projected/4ea7a10f-8208-4922-998e-0e38f321dbce-kube-api-access-v4g4f\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.444771 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.444723    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4ea7a10f-8208-4922-998e-0e38f321dbce-secret-metrics-server-client-certs\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.444771 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.444758    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4ea7a10f-8208-4922-998e-0e38f321dbce-secret-metrics-server-tls\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.445882 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.445286    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ea7a10f-8208-4922-998e-0e38f321dbce-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.446432 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.446050    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4ea7a10f-8208-4922-998e-0e38f321dbce-audit-log\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.446669 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.446633    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4ea7a10f-8208-4922-998e-0e38f321dbce-metrics-server-audit-profiles\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.449317 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.449265    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ea7a10f-8208-4922-998e-0e38f321dbce-client-ca-bundle\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.449787 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.449743    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4ea7a10f-8208-4922-998e-0e38f321dbce-secret-metrics-server-tls\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.452367 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.452312    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4ea7a10f-8208-4922-998e-0e38f321dbce-secret-metrics-server-client-certs\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.454812 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.454783    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4g4f\" (UniqueName: \"kubernetes.io/projected/4ea7a10f-8208-4922-998e-0e38f321dbce-kube-api-access-v4g4f\") pod \"metrics-server-86969d99b9-xcn44\" (UID: \"4ea7a10f-8208-4922-998e-0e38f321dbce\") " pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.471316 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.471276    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-65vb9"]
Jul 29 18:36:01.479196 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.479164    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-65vb9"
Jul 29 18:36:01.482256 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.481802    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 29 18:36:01.482256 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.482100    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-5bk2t\""
Jul 29 18:36:01.485107 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.485039    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-65vb9"]
Jul 29 18:36:01.513765 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.513718    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:01.646616 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.646579    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2bdaf43-1546-44b4-834b-810af7489214-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-65vb9\" (UID: \"b2bdaf43-1546-44b4-834b-810af7489214\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-65vb9"
Jul 29 18:36:01.646791 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.646739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfctl\" (UniqueName: \"kubernetes.io/projected/b2bdaf43-1546-44b4-834b-810af7489214-kube-api-access-dfctl\") pod \"monitoring-plugin-68b7fc9574-65vb9\" (UID: \"b2bdaf43-1546-44b4-834b-810af7489214\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-65vb9"
Jul 29 18:36:01.748226 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.748142    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2bdaf43-1546-44b4-834b-810af7489214-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-65vb9\" (UID: \"b2bdaf43-1546-44b4-834b-810af7489214\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-65vb9"
Jul 29 18:36:01.748393 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.748231    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfctl\" (UniqueName: \"kubernetes.io/projected/b2bdaf43-1546-44b4-834b-810af7489214-kube-api-access-dfctl\") pod \"monitoring-plugin-68b7fc9574-65vb9\" (UID: \"b2bdaf43-1546-44b4-834b-810af7489214\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-65vb9"
Jul 29 18:36:01.751259 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.751217    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2bdaf43-1546-44b4-834b-810af7489214-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-65vb9\" (UID: \"b2bdaf43-1546-44b4-834b-810af7489214\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-65vb9"
Jul 29 18:36:01.756977 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.756943    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfctl\" (UniqueName: \"kubernetes.io/projected/b2bdaf43-1546-44b4-834b-810af7489214-kube-api-access-dfctl\") pod \"monitoring-plugin-68b7fc9574-65vb9\" (UID: \"b2bdaf43-1546-44b4-834b-810af7489214\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-65vb9"
Jul 29 18:36:01.793247 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:01.793209    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-65vb9"
Jul 29 18:36:02.594842 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.594808    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d5f67f975-nzgwg"]
Jul 29 18:36:02.633523 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.633472    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d5f67f975-nzgwg"]
Jul 29 18:36:02.633696 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.633641    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.636934 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.636906    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 29 18:36:02.636934 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.636920    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 29 18:36:02.637177 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.637052    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 29 18:36:02.637177 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.637123    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 29 18:36:02.637879 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.637848    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-m998h\""
Jul 29 18:36:02.637996 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.637904    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 29 18:36:02.642644 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.642617    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 29 18:36:02.757220 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.757187    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-config\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.757424 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.757226    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-service-ca\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.757424 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.757266    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-trusted-ca-bundle\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.757424 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.757376    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmttp\" (UniqueName: \"kubernetes.io/projected/6a03bcd4-1c07-48e2-a377-568f4a8104db-kube-api-access-dmttp\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.757424 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.757419    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-oauth-config\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.757677 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.757476    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-oauth-serving-cert\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.757677 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.757569    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-serving-cert\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.858112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.858008    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-config\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.858112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.858062    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-service-ca\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.858112 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.858095    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-trusted-ca-bundle\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.858411 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.858187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmttp\" (UniqueName: \"kubernetes.io/projected/6a03bcd4-1c07-48e2-a377-568f4a8104db-kube-api-access-dmttp\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.858411 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.858271    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-oauth-config\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.858411 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.858316    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-oauth-serving-cert\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.858411 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.858374    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-serving-cert\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.859142 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.859079    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-service-ca\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.859252 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.859193    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-trusted-ca-bundle\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.859759 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.859726    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-oauth-serving-cert\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.860121 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.860100    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-config\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.861456 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.861429    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-serving-cert\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.861889 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.861864    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-oauth-config\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.868556 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.868471    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmttp\" (UniqueName: \"kubernetes.io/projected/6a03bcd4-1c07-48e2-a377-568f4a8104db-kube-api-access-dmttp\") pod \"console-5d5f67f975-nzgwg\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") " pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:02.939272 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.939239    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:36:02.945076 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:02.945022    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:05.709631 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:36:05.709589    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53d0338a_dbb0_4beb_8ec5_8ef3fd46b63d.slice/crio-a450fd9ed82ba8b73b3945744bb5339f2d0590380c93533b1c5236314ef52275 WatchSource:0}: Error finding container a450fd9ed82ba8b73b3945744bb5339f2d0590380c93533b1c5236314ef52275: Status 404 returned error can't find the container with id a450fd9ed82ba8b73b3945744bb5339f2d0590380c93533b1c5236314ef52275
Jul 29 18:36:05.911990 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:05.911950    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg"]
Jul 29 18:36:05.915342 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:36:05.915300    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e4c363b_bd03_40d1_b561_7be506768d71.slice/crio-d97bed52127c2ac92abad15f4d796f7aceef5f919dd0fef5a4416c37c8ae005f WatchSource:0}: Error finding container d97bed52127c2ac92abad15f4d796f7aceef5f919dd0fef5a4416c37c8ae005f: Status 404 returned error can't find the container with id d97bed52127c2ac92abad15f4d796f7aceef5f919dd0fef5a4416c37c8ae005f
Jul 29 18:36:05.945475 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:05.945395    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d5f67f975-nzgwg"]
Jul 29 18:36:05.958032 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:36:05.957863    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a03bcd4_1c07_48e2_a377_568f4a8104db.slice/crio-3e74ac018758bd364d2c1cbecb770edf0a966beb3d7472b7fe8ddd845be0bc4e WatchSource:0}: Error finding container 3e74ac018758bd364d2c1cbecb770edf0a966beb3d7472b7fe8ddd845be0bc4e: Status 404 returned error can't find the container with id 3e74ac018758bd364d2c1cbecb770edf0a966beb3d7472b7fe8ddd845be0bc4e
Jul 29 18:36:05.963906 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:05.963880    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86969d99b9-xcn44"]
Jul 29 18:36:05.966879 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:36:05.966798    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ea7a10f_8208_4922_998e_0e38f321dbce.slice/crio-434310ae48ab0ffdda9b2d724a913ac1f501a8f9993df6740cef49066b460d73 WatchSource:0}: Error finding container 434310ae48ab0ffdda9b2d724a913ac1f501a8f9993df6740cef49066b460d73: Status 404 returned error can't find the container with id 434310ae48ab0ffdda9b2d724a913ac1f501a8f9993df6740cef49066b460d73
Jul 29 18:36:06.007076 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:05.991684    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-65vb9"]
Jul 29 18:36:06.007076 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:05.993737    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"]
Jul 29 18:36:06.012712 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:36:06.012678    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2bdaf43_1546_44b4_834b_810af7489214.slice/crio-f92bcbb22d7d89ef42f6d261e21f04f2dda4e8951d4fa7549cb96589ecdaf342 WatchSource:0}: Error finding container f92bcbb22d7d89ef42f6d261e21f04f2dda4e8951d4fa7549cb96589ecdaf342: Status 404 returned error can't find the container with id f92bcbb22d7d89ef42f6d261e21f04f2dda4e8951d4fa7549cb96589ecdaf342
Jul 29 18:36:06.014020 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:36:06.013975    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf3e6f3_df70_4246_bdae_e2c16ba98c34.slice/crio-2399829f8369d07f689c5501ed5da64fd290bc8c9cd9978418b56615d5e1bd8d WatchSource:0}: Error finding container 2399829f8369d07f689c5501ed5da64fd290bc8c9cd9978418b56615d5e1bd8d: Status 404 returned error can't find the container with id 2399829f8369d07f689c5501ed5da64fd290bc8c9cd9978418b56615d5e1bd8d
Jul 29 18:36:06.076576 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:06.076478    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cc6bt" event={"ID":"b0cbd483-407c-4f29-a033-fa0beda9bba7","Type":"ContainerStarted","Data":"79047ff8f3a1d1ef2c8a0c2174b2c258422be35a1f0fb5ea0237e76e7a06bdbd"}
Jul 29 18:36:06.078308 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:06.078273    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg" event={"ID":"5e4c363b-bd03-40d1-b561-7be506768d71","Type":"ContainerStarted","Data":"cc814faa575931aeee744a7682526cc80aa94a020db418616c56e5cbbb385bdb"}
Jul 29 18:36:06.078477 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:06.078338    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg" event={"ID":"5e4c363b-bd03-40d1-b561-7be506768d71","Type":"ContainerStarted","Data":"d97bed52127c2ac92abad15f4d796f7aceef5f919dd0fef5a4416c37c8ae005f"}
Jul 29 18:36:06.079641 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:06.079613    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fq9lf" event={"ID":"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d","Type":"ContainerStarted","Data":"a450fd9ed82ba8b73b3945744bb5339f2d0590380c93533b1c5236314ef52275"}
Jul 29 18:36:06.081086 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:06.081033    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5f67f975-nzgwg" event={"ID":"6a03bcd4-1c07-48e2-a377-568f4a8104db","Type":"ContainerStarted","Data":"3e74ac018758bd364d2c1cbecb770edf0a966beb3d7472b7fe8ddd845be0bc4e"}
Jul 29 18:36:06.082450 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:06.082374    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2" event={"ID":"0cf3e6f3-df70-4246-bdae-e2c16ba98c34","Type":"ContainerStarted","Data":"2399829f8369d07f689c5501ed5da64fd290bc8c9cd9978418b56615d5e1bd8d"}
Jul 29 18:36:06.083693 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:06.083665    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-65vb9" event={"ID":"b2bdaf43-1546-44b4-834b-810af7489214","Type":"ContainerStarted","Data":"f92bcbb22d7d89ef42f6d261e21f04f2dda4e8951d4fa7549cb96589ecdaf342"}
Jul 29 18:36:06.085480 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:06.085409    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86969d99b9-xcn44" event={"ID":"4ea7a10f-8208-4922-998e-0e38f321dbce","Type":"ContainerStarted","Data":"434310ae48ab0ffdda9b2d724a913ac1f501a8f9993df6740cef49066b460d73"}
Jul 29 18:36:06.087309 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:06.087259    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-878ht" event={"ID":"9eeaeb36-0549-4b09-9663-7d77720fd5b4","Type":"ContainerStarted","Data":"e8e5b38277a20ede2e9974e33dce00df6fd9bdb5336fe040a29f5ab618a3b6de"}
Jul 29 18:36:06.087590 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:06.087548    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-878ht"
Jul 29 18:36:06.095641 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:06.095591    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-878ht"
Jul 29 18:36:06.105268 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:06.105195    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-878ht" podStartSLOduration=3.089213492 podStartE2EDuration="22.105172626s" totalImagesPullingTime="19.015959134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:36:06.103330861 +0000 UTC m=+92.054897940" watchObservedRunningTime="2026-07-29 18:36:06.105172626 +0000 UTC m=+92.056739701"
Jul 29 18:36:07.110171 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:07.110085    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg" event={"ID":"5e4c363b-bd03-40d1-b561-7be506768d71","Type":"ContainerStarted","Data":"b35fbb8a248dc3f052f802aac6dc5f84eefa1fd9c5d4cdb9ec66c35c94798472"}
Jul 29 18:36:07.753209 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:07.753009    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d548586c6-6g7fk"]
Jul 29 18:36:16.180155 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:16.180119    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cc6bt" event={"ID":"b0cbd483-407c-4f29-a033-fa0beda9bba7","Type":"ContainerStarted","Data":"3c35b9cb52452a03c93f53c2e9baad39ebb50b673a0b4b942ee8d6b5cd8f513a"}
Jul 29 18:36:16.182471 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:16.182440    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg" event={"ID":"5e4c363b-bd03-40d1-b561-7be506768d71","Type":"ContainerStarted","Data":"c9bfd51c1f36619b19789fb39870ef54b8ebd914c0490dd73720801fab50a5a0"}
Jul 29 18:36:16.184857 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:16.184826    2579 generic.go:356] "Generic (PLEG): container finished" podID="53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d" containerID="6cfe5b27f9b3a570d0fef1adf00f96f5269c4015b4190c1ef6123d2d13b8f735" exitCode=0
Jul 29 18:36:16.184997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:16.184940    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fq9lf" event={"ID":"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d","Type":"ContainerDied","Data":"6cfe5b27f9b3a570d0fef1adf00f96f5269c4015b4190c1ef6123d2d13b8f735"}
Jul 29 18:36:16.186879 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:16.186850    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5f67f975-nzgwg" event={"ID":"6a03bcd4-1c07-48e2-a377-568f4a8104db","Type":"ContainerStarted","Data":"6b246fcdbe0bb58e10f55391c83c16a19b6dc436fc690fc14386e0f3b0ff7400"}
Jul 29 18:36:16.188864 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:16.188835    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2" event={"ID":"0cf3e6f3-df70-4246-bdae-e2c16ba98c34","Type":"ContainerStarted","Data":"17aa8866a724af345f0ea4f3a6dacc59e3ba643f0da687a12adce59904dcee2e"}
Jul 29 18:36:16.189061 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:16.189042    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2" event={"ID":"0cf3e6f3-df70-4246-bdae-e2c16ba98c34","Type":"ContainerStarted","Data":"a20cfc73a0e86ee391886abf5f84d27c818310e891177dd02c2804dae92629fc"}
Jul 29 18:36:16.190689 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:16.190493    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-65vb9" event={"ID":"b2bdaf43-1546-44b4-834b-810af7489214","Type":"ContainerStarted","Data":"956c3e75e21ee40d95c518b8b9b85a2b874ce67aaebcb6634b79d235b88c4129"}
Jul 29 18:36:16.190689 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:16.190663    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-65vb9"
Jul 29 18:36:16.192602 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:16.192197    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86969d99b9-xcn44" event={"ID":"4ea7a10f-8208-4922-998e-0e38f321dbce","Type":"ContainerStarted","Data":"a5e87dc5e18f480e5542e1f5e3d432d1b628ed319eebf3b62aeb7c16d976d35e"}
Jul 29 18:36:16.197093 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:16.197064    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-65vb9"
Jul 29 18:36:16.200499 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:16.200447    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cc6bt" podStartSLOduration=34.371987648 podStartE2EDuration="1m3.200429098s" totalImagesPullingTime="28.82844145s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 18:35:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:36:16.19988348 +0000 UTC m=+102.151450579" watchObservedRunningTime="2026-07-29 18:36:16.200429098 +0000 UTC m=+102.151996175"
Jul 29 18:36:16.216887 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:16.216830    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-65vb9" podStartSLOduration=5.400458975 podStartE2EDuration="15.216805863s" totalImagesPullingTime="9.816346888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:36:16.215458311 +0000 UTC m=+102.167025380" watchObservedRunningTime="2026-07-29 18:36:16.216805863 +0000 UTC m=+102.168372944"
Jul 29 18:36:16.238655 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:16.238588    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6p9hg" podStartSLOduration=10.743027543 podStartE2EDuration="20.238565827s" totalImagesPullingTime="9.495538284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:36:16.236372375 +0000 UTC m=+102.187939465" watchObservedRunningTime="2026-07-29 18:36:16.238565827 +0000 UTC m=+102.190132906"
Jul 29 18:36:16.258743 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:16.258682    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-86969d99b9-xcn44" podStartSLOduration=5.397774006 podStartE2EDuration="15.258659887s" totalImagesPullingTime="9.860885881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:36:16.257950461 +0000 UTC m=+102.209517539" watchObservedRunningTime="2026-07-29 18:36:16.258659887 +0000 UTC m=+102.210226965"
Jul 29 18:36:16.278152 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:16.277969    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d5f67f975-nzgwg" podStartSLOduration=4.40665654 podStartE2EDuration="14.277944894s" totalImagesPullingTime="9.871288354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:36:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:36:16.276596508 +0000 UTC m=+102.228163602" watchObservedRunningTime="2026-07-29 18:36:16.277944894 +0000 UTC m=+102.229511948"
Jul 29 18:36:17.198262 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:17.198164    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fq9lf" event={"ID":"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d","Type":"ContainerStarted","Data":"f2a380f211b269823bc43f6d5dd889883b45df395e9d1ef485e00624103e8bfa"}
Jul 29 18:36:17.198262 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:17.198226    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fq9lf" event={"ID":"53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d","Type":"ContainerStarted","Data":"3c4fc4e949ce3f3bf5acc8873e210fcb064cedd72e02f3a45d561edea39a25b1"}
Jul 29 18:36:17.200492 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:17.200454    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2" event={"ID":"0cf3e6f3-df70-4246-bdae-e2c16ba98c34","Type":"ContainerStarted","Data":"328b4af43789a9afa3ce31e71c8d732560bdc2d99722a64ddc12f65983db9a77"}
Jul 29 18:36:17.221788 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:17.221726    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fq9lf" podStartSLOduration=11.217202106 podStartE2EDuration="21.221706767s" totalImagesPullingTime="9.887079568s" totalInitContainerRuntime="117.425093ms" isStatefulPod=true podCreationTimestamp="2026-07-29 18:35:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:36:17.220128458 +0000 UTC m=+103.171695533" watchObservedRunningTime="2026-07-29 18:36:17.221706767 +0000 UTC m=+103.173273844"
Jul 29 18:36:17.978712 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:17.978678    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2dp6b"
Jul 29 18:36:18.207327 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:18.207292    2579 generic.go:356] "Generic (PLEG): container finished" podID="1e7b645a-55d5-430f-a998-91b89c45b885" containerID="052d8f58a8158f41e37feeff8ea57e8956f336fe89e4c7718cd315ed448375c1" exitCode=0
Jul 29 18:36:18.207796 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:18.207361    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-pzkcg" event={"ID":"1e7b645a-55d5-430f-a998-91b89c45b885","Type":"ContainerDied","Data":"052d8f58a8158f41e37feeff8ea57e8956f336fe89e4c7718cd315ed448375c1"}
Jul 29 18:36:18.207864 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:18.207814    2579 scope.go:117] "RemoveContainer" containerID="052d8f58a8158f41e37feeff8ea57e8956f336fe89e4c7718cd315ed448375c1"
Jul 29 18:36:18.210682 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:18.210634    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2" event={"ID":"0cf3e6f3-df70-4246-bdae-e2c16ba98c34","Type":"ContainerStarted","Data":"c2d5051c56c9dde25b0c272eeff7dee0c722c9aec27806e287f8c73c0b6bc5dc"}
Jul 29 18:36:18.210826 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:18.210726    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2" event={"ID":"0cf3e6f3-df70-4246-bdae-e2c16ba98c34","Type":"ContainerStarted","Data":"3e8bff2e03d353329b11dc7e1d8b454b698fc328fe80cf2276ced6a0be4e7826"}
Jul 29 18:36:18.210826 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:18.210759    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2" event={"ID":"0cf3e6f3-df70-4246-bdae-e2c16ba98c34","Type":"ContainerStarted","Data":"5fb21f1c0aa185d8d6bbe30a8306f636d2f97714dd4474f7a664fb7f5a71b356"}
Jul 29 18:36:18.211215 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:18.211191    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:36:18.250187 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:18.250132    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2" podStartSLOduration=9.252898895 podStartE2EDuration="20.25011141s" totalImagesPullingTime="10.997212515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:35:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:36:18.24940992 +0000 UTC m=+104.200977017" watchObservedRunningTime="2026-07-29 18:36:18.25011141 +0000 UTC m=+104.201678486"
Jul 29 18:36:19.219089 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:19.219052    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-pzkcg" event={"ID":"1e7b645a-55d5-430f-a998-91b89c45b885","Type":"ContainerStarted","Data":"a1aa447e51c7ff78fcf631c637acbdb117e9ac949706e6b26341c53e182e0ba5"}
Jul 29 18:36:21.514334 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:21.514294    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:21.514778 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:21.514369    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:22.945472 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:22.945421    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:22.945884 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:22.945485    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:22.950113 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:22.950087    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:23.242078 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:23.241985    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:36:24.225242 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:24.225213    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-56dbd69f55-xnzw2"
Jul 29 18:36:32.813365 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:32.813300    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6d548586c6-6g7fk" podUID="cb702a20-d5e5-4070-b422-649738adff18" containerName="registry" containerID="cri-o://b280b9fbd822543e07155cf2338e31719d98c5a3376ece01524aa9bcdb43f01e" gracePeriod=30
Jul 29 18:36:32.934103 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:32.934066    2579 patch_prober.go:28] interesting pod/image-registry-6d548586c6-6g7fk container/registry namespace/openshift-image-registry: Readiness probe status=failure output="Get \"https://10.132.0.6:5000/healthz\": dial tcp 10.132.0.6:5000: connect: connection refused" start-of-body=
Jul 29 18:36:32.934305 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:32.934124    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6d548586c6-6g7fk" podUID="cb702a20-d5e5-4070-b422-649738adff18" containerName="registry" probeResult="failure" output="Get \"https://10.132.0.6:5000/healthz\": dial tcp 10.132.0.6:5000: connect: connection refused"
Jul 29 18:36:33.095230 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.095189    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:36:33.161717 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.161678    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cb702a20-d5e5-4070-b422-649738adff18-image-registry-private-configuration\") pod \"cb702a20-d5e5-4070-b422-649738adff18\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") "
Jul 29 18:36:33.161717 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.161719    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb702a20-d5e5-4070-b422-649738adff18-trusted-ca\") pod \"cb702a20-d5e5-4070-b422-649738adff18\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") "
Jul 29 18:36:33.162072 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.161760    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cb702a20-d5e5-4070-b422-649738adff18-installation-pull-secrets\") pod \"cb702a20-d5e5-4070-b422-649738adff18\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") "
Jul 29 18:36:33.162072 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.161780    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cb702a20-d5e5-4070-b422-649738adff18-ca-trust-extracted\") pod \"cb702a20-d5e5-4070-b422-649738adff18\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") "
Jul 29 18:36:33.162072 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.161803    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cb702a20-d5e5-4070-b422-649738adff18-registry-certificates\") pod \"cb702a20-d5e5-4070-b422-649738adff18\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") "
Jul 29 18:36:33.162072 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.161822    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-bound-sa-token\") pod \"cb702a20-d5e5-4070-b422-649738adff18\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") "
Jul 29 18:36:33.162072 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.161864    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssp5w\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-kube-api-access-ssp5w\") pod \"cb702a20-d5e5-4070-b422-649738adff18\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") "
Jul 29 18:36:33.162072 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.161891    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls\") pod \"cb702a20-d5e5-4070-b422-649738adff18\" (UID: \"cb702a20-d5e5-4070-b422-649738adff18\") "
Jul 29 18:36:33.162394 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.162358    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb702a20-d5e5-4070-b422-649738adff18-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cb702a20-d5e5-4070-b422-649738adff18" (UID: "cb702a20-d5e5-4070-b422-649738adff18"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:36:33.162394 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.162371    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb702a20-d5e5-4070-b422-649738adff18-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cb702a20-d5e5-4070-b422-649738adff18" (UID: "cb702a20-d5e5-4070-b422-649738adff18"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:36:33.164749 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.164713    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb702a20-d5e5-4070-b422-649738adff18-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cb702a20-d5e5-4070-b422-649738adff18" (UID: "cb702a20-d5e5-4070-b422-649738adff18"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:36:33.164889 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.164822    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cb702a20-d5e5-4070-b422-649738adff18" (UID: "cb702a20-d5e5-4070-b422-649738adff18"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:36:33.164889 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.164846    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb702a20-d5e5-4070-b422-649738adff18-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cb702a20-d5e5-4070-b422-649738adff18" (UID: "cb702a20-d5e5-4070-b422-649738adff18"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:36:33.164986 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.164922    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-kube-api-access-ssp5w" (OuterVolumeSpecName: "kube-api-access-ssp5w") pod "cb702a20-d5e5-4070-b422-649738adff18" (UID: "cb702a20-d5e5-4070-b422-649738adff18"). InnerVolumeSpecName "kube-api-access-ssp5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:36:33.164986 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.164958    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cb702a20-d5e5-4070-b422-649738adff18" (UID: "cb702a20-d5e5-4070-b422-649738adff18"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:36:33.170627 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.170588    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb702a20-d5e5-4070-b422-649738adff18-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cb702a20-d5e5-4070-b422-649738adff18" (UID: "cb702a20-d5e5-4070-b422-649738adff18"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:36:33.263135 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.263094    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cb702a20-d5e5-4070-b422-649738adff18-registry-certificates\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:36:33.263135 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.263126    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-bound-sa-token\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:36:33.263135 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.263139    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssp5w\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-kube-api-access-ssp5w\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:36:33.263391 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.263151    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb702a20-d5e5-4070-b422-649738adff18-registry-tls\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:36:33.263391 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.263165    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cb702a20-d5e5-4070-b422-649738adff18-image-registry-private-configuration\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:36:33.263391 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.263177    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb702a20-d5e5-4070-b422-649738adff18-trusted-ca\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:36:33.263391 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.263189    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cb702a20-d5e5-4070-b422-649738adff18-installation-pull-secrets\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:36:33.263391 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.263200    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cb702a20-d5e5-4070-b422-649738adff18-ca-trust-extracted\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:36:33.285284 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.285248    2579 generic.go:356] "Generic (PLEG): container finished" podID="cb702a20-d5e5-4070-b422-649738adff18" containerID="b280b9fbd822543e07155cf2338e31719d98c5a3376ece01524aa9bcdb43f01e" exitCode=0
Jul 29 18:36:33.285460 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.285295    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d548586c6-6g7fk"
Jul 29 18:36:33.285460 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.285328    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d548586c6-6g7fk" event={"ID":"cb702a20-d5e5-4070-b422-649738adff18","Type":"ContainerDied","Data":"b280b9fbd822543e07155cf2338e31719d98c5a3376ece01524aa9bcdb43f01e"}
Jul 29 18:36:33.285460 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.285378    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d548586c6-6g7fk" event={"ID":"cb702a20-d5e5-4070-b422-649738adff18","Type":"ContainerDied","Data":"e1557654f1c58d20a73100b647efe885e9d50ce3eef8b65f95acaa4da251d2f9"}
Jul 29 18:36:33.285460 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.285400    2579 scope.go:117] "RemoveContainer" containerID="b280b9fbd822543e07155cf2338e31719d98c5a3376ece01524aa9bcdb43f01e"
Jul 29 18:36:33.293978 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.293960    2579 scope.go:117] "RemoveContainer" containerID="b280b9fbd822543e07155cf2338e31719d98c5a3376ece01524aa9bcdb43f01e"
Jul 29 18:36:33.294273 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:36:33.294240    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b280b9fbd822543e07155cf2338e31719d98c5a3376ece01524aa9bcdb43f01e\": container with ID starting with b280b9fbd822543e07155cf2338e31719d98c5a3376ece01524aa9bcdb43f01e not found: ID does not exist" containerID="b280b9fbd822543e07155cf2338e31719d98c5a3376ece01524aa9bcdb43f01e"
Jul 29 18:36:33.294338 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.294282    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b280b9fbd822543e07155cf2338e31719d98c5a3376ece01524aa9bcdb43f01e"} err="failed to get container status \"b280b9fbd822543e07155cf2338e31719d98c5a3376ece01524aa9bcdb43f01e\": rpc error: code = NotFound desc = could not find container \"b280b9fbd822543e07155cf2338e31719d98c5a3376ece01524aa9bcdb43f01e\": container with ID starting with b280b9fbd822543e07155cf2338e31719d98c5a3376ece01524aa9bcdb43f01e not found: ID does not exist"
Jul 29 18:36:33.306027 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.305997    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d548586c6-6g7fk"]
Jul 29 18:36:33.309725 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:33.309690    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6d548586c6-6g7fk"]
Jul 29 18:36:34.588637 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:34.588609    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb702a20-d5e5-4070-b422-649738adff18" path="/var/lib/kubelet/pods/cb702a20-d5e5-4070-b422-649738adff18/volumes"
Jul 29 18:36:41.520477 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:41.520440    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:36:41.524582 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:36:41.524553    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-86969d99b9-xcn44"
Jul 29 18:37:19.257424 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.257389    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 18:37:19.258061 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.257716    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb702a20-d5e5-4070-b422-649738adff18" containerName="registry"
Jul 29 18:37:19.258061 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.257728    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb702a20-d5e5-4070-b422-649738adff18" containerName="registry"
Jul 29 18:37:19.258061 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.257786    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb702a20-d5e5-4070-b422-649738adff18" containerName="registry"
Jul 29 18:37:19.269608 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.269578    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.273387 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.273353    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 18:37:19.273577 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.273423    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 18:37:19.273577 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.273461    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 18:37:19.273712 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.273368    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 18:37:19.273790 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.273372    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-f8lrd\""
Jul 29 18:37:19.273893 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.273873    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 18:37:19.274032 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.273822    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 18:37:19.274151 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.274125    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 18:37:19.274371 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.274354    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 18:37:19.276574 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.276552    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 18:37:19.279672 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.279649    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 18:37:19.335756 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.335722    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.335756 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.335759    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7xxk\" (UniqueName: \"kubernetes.io/projected/f8a7041c-2735-41fa-a484-0c3bf995c310-kube-api-access-j7xxk\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.336005 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.335781    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.336005 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.335838    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.336005 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.335911    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.336005 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.335952    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8a7041c-2735-41fa-a484-0c3bf995c310-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.336005 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.335972    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f8a7041c-2735-41fa-a484-0c3bf995c310-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.336210 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.336011    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-config-volume\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.336210 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.336052    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-web-config\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.336210 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.336068    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8a7041c-2735-41fa-a484-0c3bf995c310-config-out\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.336210 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.336082    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8a7041c-2735-41fa-a484-0c3bf995c310-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.336210 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.336133    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.336210 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.336179    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a7041c-2735-41fa-a484-0c3bf995c310-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.436784 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.436740    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8a7041c-2735-41fa-a484-0c3bf995c310-config-out\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.436784 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.436783    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8a7041c-2735-41fa-a484-0c3bf995c310-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.437064 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.436809    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.437064 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.436830    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a7041c-2735-41fa-a484-0c3bf995c310-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.437064 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.437003    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.437064 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.437046    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7xxk\" (UniqueName: \"kubernetes.io/projected/f8a7041c-2735-41fa-a484-0c3bf995c310-kube-api-access-j7xxk\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.437268 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.437072    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.437268 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.437114    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.437268 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.437157    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.437268 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.437209    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8a7041c-2735-41fa-a484-0c3bf995c310-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.437268 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.437248    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f8a7041c-2735-41fa-a484-0c3bf995c310-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.437547 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.437289    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-config-volume\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.437547 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.437374    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-web-config\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.437668 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.437585    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8a7041c-2735-41fa-a484-0c3bf995c310-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.437866 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.437841    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a7041c-2735-41fa-a484-0c3bf995c310-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.438617 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.437859    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f8a7041c-2735-41fa-a484-0c3bf995c310-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.439863 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.439827    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8a7041c-2735-41fa-a484-0c3bf995c310-config-out\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.440010 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.439990    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.440590 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.440174    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-web-config\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.440590 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.440551    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.440835 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.440803    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8a7041c-2735-41fa-a484-0c3bf995c310-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.440911 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.440838    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.441155 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.441135    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-config-volume\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.441379 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.441356    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.442005 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.441988    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8a7041c-2735-41fa-a484-0c3bf995c310-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.447446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.447421    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7xxk\" (UniqueName: \"kubernetes.io/projected/f8a7041c-2735-41fa-a484-0c3bf995c310-kube-api-access-j7xxk\") pod \"alertmanager-main-0\" (UID: \"f8a7041c-2735-41fa-a484-0c3bf995c310\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.581942 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.581905    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 18:37:19.727615 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:19.727577    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 18:37:19.730856 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:37:19.730827    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8a7041c_2735_41fa_a484_0c3bf995c310.slice/crio-8994fafb733db8d5070d1f64da604c042d89037b3c6b3f6a543dfb38fccc78cd WatchSource:0}: Error finding container 8994fafb733db8d5070d1f64da604c042d89037b3c6b3f6a543dfb38fccc78cd: Status 404 returned error can't find the container with id 8994fafb733db8d5070d1f64da604c042d89037b3c6b3f6a543dfb38fccc78cd
Jul 29 18:37:20.475728 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:20.475675    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8a7041c-2735-41fa-a484-0c3bf995c310","Type":"ContainerStarted","Data":"8994fafb733db8d5070d1f64da604c042d89037b3c6b3f6a543dfb38fccc78cd"}
Jul 29 18:37:21.088494 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.088298    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6769746546-9s2gd"]
Jul 29 18:37:21.091150 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.091122    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.094721 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.094549    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 29 18:37:21.094721 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.094557    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 29 18:37:21.094721 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.094591    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 29 18:37:21.094721 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.094597    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-r6wtc\""
Jul 29 18:37:21.094721 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.094557    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 29 18:37:21.095098 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.094781    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 29 18:37:21.100455 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.100422    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 29 18:37:21.103466 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.103438    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6769746546-9s2gd"]
Jul 29 18:37:21.152746 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.152714    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/da9d595b-54f2-49b2-8c36-03937644edc4-telemeter-client-tls\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.152921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.152766    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da9d595b-54f2-49b2-8c36-03937644edc4-metrics-client-ca\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.152921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.152787    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da9d595b-54f2-49b2-8c36-03937644edc4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.152921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.152858    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5jst\" (UniqueName: \"kubernetes.io/projected/da9d595b-54f2-49b2-8c36-03937644edc4-kube-api-access-d5jst\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.152921 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.152914    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/da9d595b-54f2-49b2-8c36-03937644edc4-federate-client-tls\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.153119 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.152949    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/da9d595b-54f2-49b2-8c36-03937644edc4-secret-telemeter-client\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.153119 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.152984    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da9d595b-54f2-49b2-8c36-03937644edc4-serving-certs-ca-bundle\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.153119 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.153046    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da9d595b-54f2-49b2-8c36-03937644edc4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.254144 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.254102    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da9d595b-54f2-49b2-8c36-03937644edc4-metrics-client-ca\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.254413 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.254152    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da9d595b-54f2-49b2-8c36-03937644edc4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.254413 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.254187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5jst\" (UniqueName: \"kubernetes.io/projected/da9d595b-54f2-49b2-8c36-03937644edc4-kube-api-access-d5jst\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.254413 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.254222    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/da9d595b-54f2-49b2-8c36-03937644edc4-federate-client-tls\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.254413 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.254252    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/da9d595b-54f2-49b2-8c36-03937644edc4-secret-telemeter-client\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.254413 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.254283    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da9d595b-54f2-49b2-8c36-03937644edc4-serving-certs-ca-bundle\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.254413 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.254345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da9d595b-54f2-49b2-8c36-03937644edc4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.254413 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.254410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/da9d595b-54f2-49b2-8c36-03937644edc4-telemeter-client-tls\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.254991 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.254967    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da9d595b-54f2-49b2-8c36-03937644edc4-metrics-client-ca\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.255436 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.255403    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da9d595b-54f2-49b2-8c36-03937644edc4-serving-certs-ca-bundle\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.255662 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.255641    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da9d595b-54f2-49b2-8c36-03937644edc4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.257265 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.257245    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/da9d595b-54f2-49b2-8c36-03937644edc4-federate-client-tls\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.257625 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.257607    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/da9d595b-54f2-49b2-8c36-03937644edc4-secret-telemeter-client\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.257625 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.257615    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/da9d595b-54f2-49b2-8c36-03937644edc4-telemeter-client-tls\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.257703 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.257682    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da9d595b-54f2-49b2-8c36-03937644edc4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.264546 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.264478    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5jst\" (UniqueName: \"kubernetes.io/projected/da9d595b-54f2-49b2-8c36-03937644edc4-kube-api-access-d5jst\") pod \"telemeter-client-6769746546-9s2gd\" (UID: \"da9d595b-54f2-49b2-8c36-03937644edc4\") " pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.404046 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.403945    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6769746546-9s2gd"
Jul 29 18:37:21.482847 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.482810    2579 generic.go:356] "Generic (PLEG): container finished" podID="f8a7041c-2735-41fa-a484-0c3bf995c310" containerID="4863db3ccfad8e2f7c3b5545e93c6429f6f8b0e65df5cbb08e60c3b8df22dcda" exitCode=0
Jul 29 18:37:21.483244 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.482878    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8a7041c-2735-41fa-a484-0c3bf995c310","Type":"ContainerDied","Data":"4863db3ccfad8e2f7c3b5545e93c6429f6f8b0e65df5cbb08e60c3b8df22dcda"}
Jul 29 18:37:21.572717 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:21.572690    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6769746546-9s2gd"]
Jul 29 18:37:21.574799 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:37:21.574770    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda9d595b_54f2_49b2_8c36_03937644edc4.slice/crio-e3644ca64fa7bba57bef1c9d67d8146b2f5984ff67daddfe6d2f531ee98b4aab WatchSource:0}: Error finding container e3644ca64fa7bba57bef1c9d67d8146b2f5984ff67daddfe6d2f531ee98b4aab: Status 404 returned error can't find the container with id e3644ca64fa7bba57bef1c9d67d8146b2f5984ff67daddfe6d2f531ee98b4aab
Jul 29 18:37:22.489933 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:22.489895    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6769746546-9s2gd" event={"ID":"da9d595b-54f2-49b2-8c36-03937644edc4","Type":"ContainerStarted","Data":"e3644ca64fa7bba57bef1c9d67d8146b2f5984ff67daddfe6d2f531ee98b4aab"}
Jul 29 18:37:24.501439 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:24.501397    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8a7041c-2735-41fa-a484-0c3bf995c310","Type":"ContainerStarted","Data":"e8a036738620f4bb455576d9d427cc9ff01ec44dd89fc79cbe672a4378a58716"}
Jul 29 18:37:24.501913 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:24.501548    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8a7041c-2735-41fa-a484-0c3bf995c310","Type":"ContainerStarted","Data":"4e307500c362f81ffadff3b2ac0ed3b3715bae87235bb6219b8dd081a09bc648"}
Jul 29 18:37:24.501913 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:24.501584    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8a7041c-2735-41fa-a484-0c3bf995c310","Type":"ContainerStarted","Data":"cff4934a34869ecf51824657b08d0a989fb75550283c4378d412d0d4ffe75d7e"}
Jul 29 18:37:24.501913 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:24.501616    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8a7041c-2735-41fa-a484-0c3bf995c310","Type":"ContainerStarted","Data":"eb1bccee6a18c27105396203186441c44550bcd7f5e8401d73204cbfac7b5499"}
Jul 29 18:37:24.501913 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:24.501650    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8a7041c-2735-41fa-a484-0c3bf995c310","Type":"ContainerStarted","Data":"6635f35aa13f1fd6ff9f0c8f0c251ee2b981a13388b6978abc0cf61ee808fcae"}
Jul 29 18:37:24.501913 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:24.501684    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8a7041c-2735-41fa-a484-0c3bf995c310","Type":"ContainerStarted","Data":"ae607d5e8a15311f2c80cbdac97e6894a1fad1240a708debd309566178912d2f"}
Jul 29 18:37:24.503445 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:24.503414    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6769746546-9s2gd" event={"ID":"da9d595b-54f2-49b2-8c36-03937644edc4","Type":"ContainerStarted","Data":"a50cc89bb38be00c7a7244811f13201fe6c4afb42a81e6f2df3b39b8bb25002c"}
Jul 29 18:37:24.503623 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:24.503465    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6769746546-9s2gd" event={"ID":"da9d595b-54f2-49b2-8c36-03937644edc4","Type":"ContainerStarted","Data":"71df8b6ca3bd025d97ab20ed673fdad77cce85e606a2a26a3ac530fc6fb55219"}
Jul 29 18:37:24.503623 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:24.503491    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6769746546-9s2gd" event={"ID":"da9d595b-54f2-49b2-8c36-03937644edc4","Type":"ContainerStarted","Data":"8ca61f8ef3ae6ec58f36ed28d77d00cdd8a9db683ea97f4ce0cfa334b2e682e8"}
Jul 29 18:37:24.532772 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:24.532717    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.474125473 podStartE2EDuration="5.532699717s" totalImagesPullingTime="3.044485658s" totalInitContainerRuntime="14.088586ms" isStatefulPod=false podCreationTimestamp="2026-07-29 18:37:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:37:24.530567628 +0000 UTC m=+170.482134705" watchObservedRunningTime="2026-07-29 18:37:24.532699717 +0000 UTC m=+170.484266794"
Jul 29 18:37:24.555838 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:24.555761    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6769746546-9s2gd" podStartSLOduration=1.687196563 podStartE2EDuration="3.555739308s" totalImagesPullingTime="1.868542745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:37:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:37:24.553390681 +0000 UTC m=+170.504957755" watchObservedRunningTime="2026-07-29 18:37:24.555739308 +0000 UTC m=+170.507306583"
Jul 29 18:37:25.405442 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.405400    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78f4bb65b8-fsxfb"]
Jul 29 18:37:25.408403 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.408374    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.422055 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.422017    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78f4bb65b8-fsxfb"]
Jul 29 18:37:25.498059 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.497516    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt2p6\" (UniqueName: \"kubernetes.io/projected/595821ec-e120-4e99-bad1-ad5c4d48c001-kube-api-access-vt2p6\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.498059 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.497600    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-trusted-ca-bundle\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.498059 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.497677    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/595821ec-e120-4e99-bad1-ad5c4d48c001-console-oauth-config\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.498059 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.497705    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-oauth-serving-cert\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.498059 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.497743    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-service-ca\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.498059 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.497797    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/595821ec-e120-4e99-bad1-ad5c4d48c001-console-serving-cert\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.498059 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.497859    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-console-config\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.598756 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.598651    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-service-ca\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.599225 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.599026    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/595821ec-e120-4e99-bad1-ad5c4d48c001-console-serving-cert\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.599225 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.599127    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-console-config\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.599225 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.599213    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt2p6\" (UniqueName: \"kubernetes.io/projected/595821ec-e120-4e99-bad1-ad5c4d48c001-kube-api-access-vt2p6\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.599381 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.599274    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-trusted-ca-bundle\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.599433 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.599417    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/595821ec-e120-4e99-bad1-ad5c4d48c001-console-oauth-config\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.599482 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.599439    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-service-ca\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.599565 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.599527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-oauth-serving-cert\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.600249 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.600216    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-oauth-serving-cert\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.600572 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.600548    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-trusted-ca-bundle\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.601321 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.601294    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-console-config\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.603430 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.603401    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/595821ec-e120-4e99-bad1-ad5c4d48c001-console-serving-cert\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.604041 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.604010    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/595821ec-e120-4e99-bad1-ad5c4d48c001-console-oauth-config\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.609422 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.609396    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt2p6\" (UniqueName: \"kubernetes.io/projected/595821ec-e120-4e99-bad1-ad5c4d48c001-kube-api-access-vt2p6\") pod \"console-78f4bb65b8-fsxfb\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") " pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.724418 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.724316    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:25.862482 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:25.862449    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78f4bb65b8-fsxfb"]
Jul 29 18:37:25.864188 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:37:25.864155    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod595821ec_e120_4e99_bad1_ad5c4d48c001.slice/crio-553a7e4ea280281b9356761c2d2819ffe8c59dc2d5876b0cd9a3174864d2751a WatchSource:0}: Error finding container 553a7e4ea280281b9356761c2d2819ffe8c59dc2d5876b0cd9a3174864d2751a: Status 404 returned error can't find the container with id 553a7e4ea280281b9356761c2d2819ffe8c59dc2d5876b0cd9a3174864d2751a
Jul 29 18:37:26.515910 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:26.515869    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78f4bb65b8-fsxfb" event={"ID":"595821ec-e120-4e99-bad1-ad5c4d48c001","Type":"ContainerStarted","Data":"56ce965beae6bdc72c3a6870fe58b19355ff54bc5320a778ad2d529ce7424fcc"}
Jul 29 18:37:26.516128 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:26.515920    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78f4bb65b8-fsxfb" event={"ID":"595821ec-e120-4e99-bad1-ad5c4d48c001","Type":"ContainerStarted","Data":"553a7e4ea280281b9356761c2d2819ffe8c59dc2d5876b0cd9a3174864d2751a"}
Jul 29 18:37:26.532582 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:26.532498    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78f4bb65b8-fsxfb" podStartSLOduration=1.532475885 podStartE2EDuration="1.532475885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:37:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:37:26.532421534 +0000 UTC m=+172.483988611" watchObservedRunningTime="2026-07-29 18:37:26.532475885 +0000 UTC m=+172.484042963"
Jul 29 18:37:35.725338 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:35.725294    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:35.725771 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:35.725382    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:35.730314 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:35.730284    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:36.570313 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:36.570284    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:37:36.621534 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:37:36.621481    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d5f67f975-nzgwg"]
Jul 29 18:38:01.643485 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.643422    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d5f67f975-nzgwg" podUID="6a03bcd4-1c07-48e2-a377-568f4a8104db" containerName="console" containerID="cri-o://6b246fcdbe0bb58e10f55391c83c16a19b6dc436fc690fc14386e0f3b0ff7400" gracePeriod=15
Jul 29 18:38:01.898225 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.898152    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:38:01.938024 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.937989    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-oauth-serving-cert\") pod \"6a03bcd4-1c07-48e2-a377-568f4a8104db\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") "
Jul 29 18:38:01.938192 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.938060    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-service-ca\") pod \"6a03bcd4-1c07-48e2-a377-568f4a8104db\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") "
Jul 29 18:38:01.938192 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.938100    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-config\") pod \"6a03bcd4-1c07-48e2-a377-568f4a8104db\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") "
Jul 29 18:38:01.938192 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.938142    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-trusted-ca-bundle\") pod \"6a03bcd4-1c07-48e2-a377-568f4a8104db\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") "
Jul 29 18:38:01.938347 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.938320    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmttp\" (UniqueName: \"kubernetes.io/projected/6a03bcd4-1c07-48e2-a377-568f4a8104db-kube-api-access-dmttp\") pod \"6a03bcd4-1c07-48e2-a377-568f4a8104db\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") "
Jul 29 18:38:01.938397 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.938353    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6a03bcd4-1c07-48e2-a377-568f4a8104db" (UID: "6a03bcd4-1c07-48e2-a377-568f4a8104db"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:38:01.938397 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.938387    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-oauth-config\") pod \"6a03bcd4-1c07-48e2-a377-568f4a8104db\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") "
Jul 29 18:38:01.938480 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.938429    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-serving-cert\") pod \"6a03bcd4-1c07-48e2-a377-568f4a8104db\" (UID: \"6a03bcd4-1c07-48e2-a377-568f4a8104db\") "
Jul 29 18:38:01.938568 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.938541    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-service-ca" (OuterVolumeSpecName: "service-ca") pod "6a03bcd4-1c07-48e2-a377-568f4a8104db" (UID: "6a03bcd4-1c07-48e2-a377-568f4a8104db"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:38:01.938568 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.938553    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-config" (OuterVolumeSpecName: "console-config") pod "6a03bcd4-1c07-48e2-a377-568f4a8104db" (UID: "6a03bcd4-1c07-48e2-a377-568f4a8104db"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:38:01.938704 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.938577    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6a03bcd4-1c07-48e2-a377-568f4a8104db" (UID: "6a03bcd4-1c07-48e2-a377-568f4a8104db"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:38:01.938784 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.938762    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-service-ca\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:38:01.938842 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.938791    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:38:01.938842 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.938803    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-trusted-ca-bundle\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:38:01.938842 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.938832    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a03bcd4-1c07-48e2-a377-568f4a8104db-oauth-serving-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:38:01.940716 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.940679    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6a03bcd4-1c07-48e2-a377-568f4a8104db" (UID: "6a03bcd4-1c07-48e2-a377-568f4a8104db"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:38:01.940823 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.940743    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6a03bcd4-1c07-48e2-a377-568f4a8104db" (UID: "6a03bcd4-1c07-48e2-a377-568f4a8104db"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:38:01.940823 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:01.940766    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a03bcd4-1c07-48e2-a377-568f4a8104db-kube-api-access-dmttp" (OuterVolumeSpecName: "kube-api-access-dmttp") pod "6a03bcd4-1c07-48e2-a377-568f4a8104db" (UID: "6a03bcd4-1c07-48e2-a377-568f4a8104db"). InnerVolumeSpecName "kube-api-access-dmttp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:38:02.039720 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:02.039678    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmttp\" (UniqueName: \"kubernetes.io/projected/6a03bcd4-1c07-48e2-a377-568f4a8104db-kube-api-access-dmttp\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:38:02.039720 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:02.039708    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-oauth-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:38:02.039720 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:02.039717    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a03bcd4-1c07-48e2-a377-568f4a8104db-console-serving-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:38:02.689871 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:02.689835    2579 generic.go:356] "Generic (PLEG): container finished" podID="6a03bcd4-1c07-48e2-a377-568f4a8104db" containerID="6b246fcdbe0bb58e10f55391c83c16a19b6dc436fc690fc14386e0f3b0ff7400" exitCode=0
Jul 29 18:38:02.690351 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:02.689911    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5f67f975-nzgwg"
Jul 29 18:38:02.690351 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:02.689918    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5f67f975-nzgwg" event={"ID":"6a03bcd4-1c07-48e2-a377-568f4a8104db","Type":"ContainerDied","Data":"6b246fcdbe0bb58e10f55391c83c16a19b6dc436fc690fc14386e0f3b0ff7400"}
Jul 29 18:38:02.690351 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:02.690024    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5f67f975-nzgwg" event={"ID":"6a03bcd4-1c07-48e2-a377-568f4a8104db","Type":"ContainerDied","Data":"3e74ac018758bd364d2c1cbecb770edf0a966beb3d7472b7fe8ddd845be0bc4e"}
Jul 29 18:38:02.690351 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:02.690059    2579 scope.go:117] "RemoveContainer" containerID="6b246fcdbe0bb58e10f55391c83c16a19b6dc436fc690fc14386e0f3b0ff7400"
Jul 29 18:38:02.698761 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:02.698737    2579 scope.go:117] "RemoveContainer" containerID="6b246fcdbe0bb58e10f55391c83c16a19b6dc436fc690fc14386e0f3b0ff7400"
Jul 29 18:38:02.699053 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:38:02.699030    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b246fcdbe0bb58e10f55391c83c16a19b6dc436fc690fc14386e0f3b0ff7400\": container with ID starting with 6b246fcdbe0bb58e10f55391c83c16a19b6dc436fc690fc14386e0f3b0ff7400 not found: ID does not exist" containerID="6b246fcdbe0bb58e10f55391c83c16a19b6dc436fc690fc14386e0f3b0ff7400"
Jul 29 18:38:02.699156 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:02.699060    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b246fcdbe0bb58e10f55391c83c16a19b6dc436fc690fc14386e0f3b0ff7400"} err="failed to get container status \"6b246fcdbe0bb58e10f55391c83c16a19b6dc436fc690fc14386e0f3b0ff7400\": rpc error: code = NotFound desc = could not find container \"6b246fcdbe0bb58e10f55391c83c16a19b6dc436fc690fc14386e0f3b0ff7400\": container with ID starting with 6b246fcdbe0bb58e10f55391c83c16a19b6dc436fc690fc14386e0f3b0ff7400 not found: ID does not exist"
Jul 29 18:38:02.708612 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:02.708576    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d5f67f975-nzgwg"]
Jul 29 18:38:02.713288 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:02.713260    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d5f67f975-nzgwg"]
Jul 29 18:38:04.584816 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:04.584785    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a03bcd4-1c07-48e2-a377-568f4a8104db" path="/var/lib/kubelet/pods/6a03bcd4-1c07-48e2-a377-568f4a8104db/volumes"
Jul 29 18:38:33.467606 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.467570    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8dc7bc744-x87d5"]
Jul 29 18:38:33.468082 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.467919    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a03bcd4-1c07-48e2-a377-568f4a8104db" containerName="console"
Jul 29 18:38:33.468082 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.467931    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a03bcd4-1c07-48e2-a377-568f4a8104db" containerName="console"
Jul 29 18:38:33.468082 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.467997    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a03bcd4-1c07-48e2-a377-568f4a8104db" containerName="console"
Jul 29 18:38:33.470996 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.470973    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.499623 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.499585    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8dc7bc744-x87d5"]
Jul 29 18:38:33.618971 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.618928    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92pd4\" (UniqueName: \"kubernetes.io/projected/2359f584-67d9-4b8f-a2e4-86af703088fd-kube-api-access-92pd4\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.619149 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.618997    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-service-ca\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.619149 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.619033    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-trusted-ca-bundle\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.619149 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.619050    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-oauth-serving-cert\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.619149 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.619076    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-console-config\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.619283 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.619206    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2359f584-67d9-4b8f-a2e4-86af703088fd-console-oauth-config\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.619283 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.619255    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2359f584-67d9-4b8f-a2e4-86af703088fd-console-serving-cert\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.720653 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.720560    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2359f584-67d9-4b8f-a2e4-86af703088fd-console-oauth-config\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.720653 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.720607    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2359f584-67d9-4b8f-a2e4-86af703088fd-console-serving-cert\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.720653 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.720642    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92pd4\" (UniqueName: \"kubernetes.io/projected/2359f584-67d9-4b8f-a2e4-86af703088fd-kube-api-access-92pd4\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.721015 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.720682    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-service-ca\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.721015 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.720703    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-trusted-ca-bundle\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.721015 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.720722    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-oauth-serving-cert\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.721015 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.720756    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-console-config\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.721497 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.721471    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-service-ca\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.721622 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.721600    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-oauth-serving-cert\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.721622 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.721612    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-console-config\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.721895 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.721878    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-trusted-ca-bundle\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.723145 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.723124    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2359f584-67d9-4b8f-a2e4-86af703088fd-console-oauth-config\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.723326 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.723310    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2359f584-67d9-4b8f-a2e4-86af703088fd-console-serving-cert\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.728957 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.728927    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92pd4\" (UniqueName: \"kubernetes.io/projected/2359f584-67d9-4b8f-a2e4-86af703088fd-kube-api-access-92pd4\") pod \"console-8dc7bc744-x87d5\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") " pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.780466 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.780429    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:33.922802 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:33.922759    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8dc7bc744-x87d5"]
Jul 29 18:38:33.926406 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:38:33.926369    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2359f584_67d9_4b8f_a2e4_86af703088fd.slice/crio-92c475e7cda7305f1bd9c1f57ea1347f93e611940bddee8fc6c65d6c3a350519 WatchSource:0}: Error finding container 92c475e7cda7305f1bd9c1f57ea1347f93e611940bddee8fc6c65d6c3a350519: Status 404 returned error can't find the container with id 92c475e7cda7305f1bd9c1f57ea1347f93e611940bddee8fc6c65d6c3a350519
Jul 29 18:38:34.837092 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:34.837047    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8dc7bc744-x87d5" event={"ID":"2359f584-67d9-4b8f-a2e4-86af703088fd","Type":"ContainerStarted","Data":"69cf5323bb8eaa8501e823c6fb12671fac29813e8086c7c8a38141bfeb24ed62"}
Jul 29 18:38:34.837556 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:34.837107    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8dc7bc744-x87d5" event={"ID":"2359f584-67d9-4b8f-a2e4-86af703088fd","Type":"ContainerStarted","Data":"92c475e7cda7305f1bd9c1f57ea1347f93e611940bddee8fc6c65d6c3a350519"}
Jul 29 18:38:34.856956 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:34.856897    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8dc7bc744-x87d5" podStartSLOduration=1.856879557 podStartE2EDuration="1.856879557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:38:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:38:34.85470363 +0000 UTC m=+240.806270729" watchObservedRunningTime="2026-07-29 18:38:34.856879557 +0000 UTC m=+240.808446699"
Jul 29 18:38:43.781617 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:43.781565    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:43.782042 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:43.781648    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:43.786531 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:43.786480    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:43.882187 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:43.882157    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:38:43.927954 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:38:43.927915    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78f4bb65b8-fsxfb"]
Jul 29 18:39:08.949466 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:08.949400    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78f4bb65b8-fsxfb" podUID="595821ec-e120-4e99-bad1-ad5c4d48c001" containerName="console" containerID="cri-o://56ce965beae6bdc72c3a6870fe58b19355ff54bc5320a778ad2d529ce7424fcc" gracePeriod=15
Jul 29 18:39:09.202446 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.202376    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:39:09.342910 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.342874    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-oauth-serving-cert\") pod \"595821ec-e120-4e99-bad1-ad5c4d48c001\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") "
Jul 29 18:39:09.343103 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.342948    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-console-config\") pod \"595821ec-e120-4e99-bad1-ad5c4d48c001\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") "
Jul 29 18:39:09.343103 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.342999    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vt2p6\" (UniqueName: \"kubernetes.io/projected/595821ec-e120-4e99-bad1-ad5c4d48c001-kube-api-access-vt2p6\") pod \"595821ec-e120-4e99-bad1-ad5c4d48c001\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") "
Jul 29 18:39:09.343103 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.343025    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/595821ec-e120-4e99-bad1-ad5c4d48c001-console-oauth-config\") pod \"595821ec-e120-4e99-bad1-ad5c4d48c001\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") "
Jul 29 18:39:09.343103 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.343052    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-trusted-ca-bundle\") pod \"595821ec-e120-4e99-bad1-ad5c4d48c001\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") "
Jul 29 18:39:09.343103 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.343092    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-service-ca\") pod \"595821ec-e120-4e99-bad1-ad5c4d48c001\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") "
Jul 29 18:39:09.343334 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.343122    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/595821ec-e120-4e99-bad1-ad5c4d48c001-console-serving-cert\") pod \"595821ec-e120-4e99-bad1-ad5c4d48c001\" (UID: \"595821ec-e120-4e99-bad1-ad5c4d48c001\") "
Jul 29 18:39:09.343334 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.343312    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-console-config" (OuterVolumeSpecName: "console-config") pod "595821ec-e120-4e99-bad1-ad5c4d48c001" (UID: "595821ec-e120-4e99-bad1-ad5c4d48c001"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:39:09.343591 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.343566    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-service-ca" (OuterVolumeSpecName: "service-ca") pod "595821ec-e120-4e99-bad1-ad5c4d48c001" (UID: "595821ec-e120-4e99-bad1-ad5c4d48c001"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:39:09.343591 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.343590    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-console-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:39:09.343761 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.343602    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "595821ec-e120-4e99-bad1-ad5c4d48c001" (UID: "595821ec-e120-4e99-bad1-ad5c4d48c001"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:39:09.343761 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.343652    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "595821ec-e120-4e99-bad1-ad5c4d48c001" (UID: "595821ec-e120-4e99-bad1-ad5c4d48c001"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:39:09.345322 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.345298    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/595821ec-e120-4e99-bad1-ad5c4d48c001-kube-api-access-vt2p6" (OuterVolumeSpecName: "kube-api-access-vt2p6") pod "595821ec-e120-4e99-bad1-ad5c4d48c001" (UID: "595821ec-e120-4e99-bad1-ad5c4d48c001"). InnerVolumeSpecName "kube-api-access-vt2p6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:39:09.345322 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.345309    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/595821ec-e120-4e99-bad1-ad5c4d48c001-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "595821ec-e120-4e99-bad1-ad5c4d48c001" (UID: "595821ec-e120-4e99-bad1-ad5c4d48c001"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:39:09.345423 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.345364    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/595821ec-e120-4e99-bad1-ad5c4d48c001-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "595821ec-e120-4e99-bad1-ad5c4d48c001" (UID: "595821ec-e120-4e99-bad1-ad5c4d48c001"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:39:09.444193 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.444140    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-oauth-serving-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:39:09.444193 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.444184    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt2p6\" (UniqueName: \"kubernetes.io/projected/595821ec-e120-4e99-bad1-ad5c4d48c001-kube-api-access-vt2p6\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:39:09.444193 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.444195    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/595821ec-e120-4e99-bad1-ad5c4d48c001-console-oauth-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:39:09.444193 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.444203    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-trusted-ca-bundle\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:39:09.444193 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.444212    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/595821ec-e120-4e99-bad1-ad5c4d48c001-service-ca\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:39:09.444492 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:09.444223    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/595821ec-e120-4e99-bad1-ad5c4d48c001-console-serving-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:39:10.005942 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:10.005906    2579 generic.go:356] "Generic (PLEG): container finished" podID="595821ec-e120-4e99-bad1-ad5c4d48c001" containerID="56ce965beae6bdc72c3a6870fe58b19355ff54bc5320a778ad2d529ce7424fcc" exitCode=0
Jul 29 18:39:10.006409 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:10.005951    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78f4bb65b8-fsxfb"
Jul 29 18:39:10.006409 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:10.005960    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78f4bb65b8-fsxfb" event={"ID":"595821ec-e120-4e99-bad1-ad5c4d48c001","Type":"ContainerDied","Data":"56ce965beae6bdc72c3a6870fe58b19355ff54bc5320a778ad2d529ce7424fcc"}
Jul 29 18:39:10.006409 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:10.006001    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78f4bb65b8-fsxfb" event={"ID":"595821ec-e120-4e99-bad1-ad5c4d48c001","Type":"ContainerDied","Data":"553a7e4ea280281b9356761c2d2819ffe8c59dc2d5876b0cd9a3174864d2751a"}
Jul 29 18:39:10.006409 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:10.006020    2579 scope.go:117] "RemoveContainer" containerID="56ce965beae6bdc72c3a6870fe58b19355ff54bc5320a778ad2d529ce7424fcc"
Jul 29 18:39:10.014749 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:10.014728    2579 scope.go:117] "RemoveContainer" containerID="56ce965beae6bdc72c3a6870fe58b19355ff54bc5320a778ad2d529ce7424fcc"
Jul 29 18:39:10.015063 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:39:10.015040    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56ce965beae6bdc72c3a6870fe58b19355ff54bc5320a778ad2d529ce7424fcc\": container with ID starting with 56ce965beae6bdc72c3a6870fe58b19355ff54bc5320a778ad2d529ce7424fcc not found: ID does not exist" containerID="56ce965beae6bdc72c3a6870fe58b19355ff54bc5320a778ad2d529ce7424fcc"
Jul 29 18:39:10.015108 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:10.015072    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56ce965beae6bdc72c3a6870fe58b19355ff54bc5320a778ad2d529ce7424fcc"} err="failed to get container status \"56ce965beae6bdc72c3a6870fe58b19355ff54bc5320a778ad2d529ce7424fcc\": rpc error: code = NotFound desc = could not find container \"56ce965beae6bdc72c3a6870fe58b19355ff54bc5320a778ad2d529ce7424fcc\": container with ID starting with 56ce965beae6bdc72c3a6870fe58b19355ff54bc5320a778ad2d529ce7424fcc not found: ID does not exist"
Jul 29 18:39:10.028533 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:10.028489    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78f4bb65b8-fsxfb"]
Jul 29 18:39:10.032939 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:10.032912    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78f4bb65b8-fsxfb"]
Jul 29 18:39:10.584244 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:10.584205    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="595821ec-e120-4e99-bad1-ad5c4d48c001" path="/var/lib/kubelet/pods/595821ec-e120-4e99-bad1-ad5c4d48c001/volumes"
Jul 29 18:39:34.410941 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:34.410908    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 18:39:34.414280 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:34.414253    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 18:39:34.421002 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:39:34.420975    2579 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 18:40:09.931760 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:09.931724    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd"]
Jul 29 18:40:09.934166 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:09.932062    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="595821ec-e120-4e99-bad1-ad5c4d48c001" containerName="console"
Jul 29 18:40:09.934166 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:09.932074    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="595821ec-e120-4e99-bad1-ad5c4d48c001" containerName="console"
Jul 29 18:40:09.934166 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:09.932164    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="595821ec-e120-4e99-bad1-ad5c4d48c001" containerName="console"
Jul 29 18:40:09.935039 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:09.935019    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd"
Jul 29 18:40:09.938313 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:09.938287    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 29 18:40:09.938313 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:09.938309    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 29 18:40:09.938562 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:09.938385    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-rb8sg\""
Jul 29 18:40:09.939627 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:09.939611    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 29 18:40:09.951668 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:09.951634    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd"]
Jul 29 18:40:10.068212 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:10.068175    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxhqt\" (UniqueName: \"kubernetes.io/projected/c639d202-b019-484a-8033-2055eb4c4beb-kube-api-access-kxhqt\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd\" (UID: \"c639d202-b019-484a-8033-2055eb4c4beb\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd"
Jul 29 18:40:10.068212 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:10.068220    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/c639d202-b019-484a-8033-2055eb4c4beb-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd\" (UID: \"c639d202-b019-484a-8033-2055eb4c4beb\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd"
Jul 29 18:40:10.169402 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:10.169366    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxhqt\" (UniqueName: \"kubernetes.io/projected/c639d202-b019-484a-8033-2055eb4c4beb-kube-api-access-kxhqt\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd\" (UID: \"c639d202-b019-484a-8033-2055eb4c4beb\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd"
Jul 29 18:40:10.169557 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:10.169409    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/c639d202-b019-484a-8033-2055eb4c4beb-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd\" (UID: \"c639d202-b019-484a-8033-2055eb4c4beb\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd"
Jul 29 18:40:10.171927 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:10.171893    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/c639d202-b019-484a-8033-2055eb4c4beb-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd\" (UID: \"c639d202-b019-484a-8033-2055eb4c4beb\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd"
Jul 29 18:40:10.190058 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:10.189989    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxhqt\" (UniqueName: \"kubernetes.io/projected/c639d202-b019-484a-8033-2055eb4c4beb-kube-api-access-kxhqt\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd\" (UID: \"c639d202-b019-484a-8033-2055eb4c4beb\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd"
Jul 29 18:40:10.245396 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:10.245360    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd"
Jul 29 18:40:10.398902 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:10.398839    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd"]
Jul 29 18:40:10.401427 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:40:10.401391    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc639d202_b019_484a_8033_2055eb4c4beb.slice/crio-cf2189b2a5357ead51676439fadfd17611f9360d34d2713eb8c159347af95fbd WatchSource:0}: Error finding container cf2189b2a5357ead51676439fadfd17611f9360d34d2713eb8c159347af95fbd: Status 404 returned error can't find the container with id cf2189b2a5357ead51676439fadfd17611f9360d34d2713eb8c159347af95fbd
Jul 29 18:40:10.403258 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:10.403239    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:40:11.277209 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:11.277166    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd" event={"ID":"c639d202-b019-484a-8033-2055eb4c4beb","Type":"ContainerStarted","Data":"cf2189b2a5357ead51676439fadfd17611f9360d34d2713eb8c159347af95fbd"}
Jul 29 18:40:14.297099 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:14.297053    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd" event={"ID":"c639d202-b019-484a-8033-2055eb4c4beb","Type":"ContainerStarted","Data":"57c42f9bdf58ebf035e5f13fd483486d35144a814200f938078a8528cadfa49a"}
Jul 29 18:40:14.297606 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:14.297157    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd"
Jul 29 18:40:14.317715 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:14.317652    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd" podStartSLOduration=2.112892195 podStartE2EDuration="5.317638133s" totalImagesPullingTime="3.204745938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:40:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:40:14.316353044 +0000 UTC m=+340.267920120" watchObservedRunningTime="2026-07-29 18:40:14.317638133 +0000 UTC m=+340.269205204"
Jul 29 18:40:15.123124 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:15.123090    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-c7lmx"]
Jul 29 18:40:15.126441 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:15.126422    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-c7lmx"
Jul 29 18:40:15.129264 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:15.129235    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 29 18:40:15.129264 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:15.129253    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 29 18:40:15.129435 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:15.129281    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-tvzdv\""
Jul 29 18:40:15.139081 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:15.139040    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-c7lmx"]
Jul 29 18:40:15.317992 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:15.317952    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a01dba19-973d-4f8c-a96b-451de77c3a39-certificates\") pod \"keda-admission-56f9798bb4-c7lmx\" (UID: \"a01dba19-973d-4f8c-a96b-451de77c3a39\") " pod="openshift-keda/keda-admission-56f9798bb4-c7lmx"
Jul 29 18:40:15.318485 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:15.318006    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4vrf\" (UniqueName: \"kubernetes.io/projected/a01dba19-973d-4f8c-a96b-451de77c3a39-kube-api-access-t4vrf\") pod \"keda-admission-56f9798bb4-c7lmx\" (UID: \"a01dba19-973d-4f8c-a96b-451de77c3a39\") " pod="openshift-keda/keda-admission-56f9798bb4-c7lmx"
Jul 29 18:40:15.419672 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:15.419576    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a01dba19-973d-4f8c-a96b-451de77c3a39-certificates\") pod \"keda-admission-56f9798bb4-c7lmx\" (UID: \"a01dba19-973d-4f8c-a96b-451de77c3a39\") " pod="openshift-keda/keda-admission-56f9798bb4-c7lmx"
Jul 29 18:40:15.419897 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:15.419878    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4vrf\" (UniqueName: \"kubernetes.io/projected/a01dba19-973d-4f8c-a96b-451de77c3a39-kube-api-access-t4vrf\") pod \"keda-admission-56f9798bb4-c7lmx\" (UID: \"a01dba19-973d-4f8c-a96b-451de77c3a39\") " pod="openshift-keda/keda-admission-56f9798bb4-c7lmx"
Jul 29 18:40:15.422911 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:15.422882    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a01dba19-973d-4f8c-a96b-451de77c3a39-certificates\") pod \"keda-admission-56f9798bb4-c7lmx\" (UID: \"a01dba19-973d-4f8c-a96b-451de77c3a39\") " pod="openshift-keda/keda-admission-56f9798bb4-c7lmx"
Jul 29 18:40:15.429353 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:15.429318    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4vrf\" (UniqueName: \"kubernetes.io/projected/a01dba19-973d-4f8c-a96b-451de77c3a39-kube-api-access-t4vrf\") pod \"keda-admission-56f9798bb4-c7lmx\" (UID: \"a01dba19-973d-4f8c-a96b-451de77c3a39\") " pod="openshift-keda/keda-admission-56f9798bb4-c7lmx"
Jul 29 18:40:15.437051 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:15.437007    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-c7lmx"
Jul 29 18:40:15.595476 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:15.595448    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-c7lmx"]
Jul 29 18:40:16.310546 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:16.310492    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-c7lmx" event={"ID":"a01dba19-973d-4f8c-a96b-451de77c3a39","Type":"ContainerStarted","Data":"4147cf5a5a39c31fcab40727ead514a6f9b2ba07c54633de757fbbb7dc869eea"}
Jul 29 18:40:17.316820 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:17.316784    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-c7lmx" event={"ID":"a01dba19-973d-4f8c-a96b-451de77c3a39","Type":"ContainerStarted","Data":"9d3b2d5a8dcff4be8945ea2890c41926dedc90612765b1cd61c49296cd9ad7f8"}
Jul 29 18:40:17.317280 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:17.316921    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-56f9798bb4-c7lmx"
Jul 29 18:40:17.335409 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:17.335358    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-56f9798bb4-c7lmx" podStartSLOduration=0.993385521 podStartE2EDuration="2.335314543s" totalImagesPullingTime="1.341929022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:40:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:40:17.334319654 +0000 UTC m=+343.285886760" watchObservedRunningTime="2026-07-29 18:40:17.335314543 +0000 UTC m=+343.286881653"
Jul 29 18:40:35.305956 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:35.305881    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-cpbhd"
Jul 29 18:40:38.325151 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:40:38.325120    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-56f9798bb4-c7lmx"
Jul 29 18:41:35.570236 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:35.570204    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vw45w"]
Jul 29 18:41:35.573738 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:35.573717    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-vw45w"
Jul 29 18:41:35.576278 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:35.576253    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 29 18:41:35.576392 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:35.576301    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ngspf\""
Jul 29 18:41:35.577111 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:35.577092    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 29 18:41:35.583333 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:35.583306    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vw45w"]
Jul 29 18:41:35.691402 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:35.691357    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/466388a3-4cba-4a78-a705-276c03be16fa-bound-sa-token\") pod \"cert-manager-55b4d954d5-vw45w\" (UID: \"466388a3-4cba-4a78-a705-276c03be16fa\") " pod="cert-manager/cert-manager-55b4d954d5-vw45w"
Jul 29 18:41:35.691631 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:35.691494    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vzqd\" (UniqueName: \"kubernetes.io/projected/466388a3-4cba-4a78-a705-276c03be16fa-kube-api-access-6vzqd\") pod \"cert-manager-55b4d954d5-vw45w\" (UID: \"466388a3-4cba-4a78-a705-276c03be16fa\") " pod="cert-manager/cert-manager-55b4d954d5-vw45w"
Jul 29 18:41:35.792572 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:35.792497    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vzqd\" (UniqueName: \"kubernetes.io/projected/466388a3-4cba-4a78-a705-276c03be16fa-kube-api-access-6vzqd\") pod \"cert-manager-55b4d954d5-vw45w\" (UID: \"466388a3-4cba-4a78-a705-276c03be16fa\") " pod="cert-manager/cert-manager-55b4d954d5-vw45w"
Jul 29 18:41:35.792766 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:35.792618    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/466388a3-4cba-4a78-a705-276c03be16fa-bound-sa-token\") pod \"cert-manager-55b4d954d5-vw45w\" (UID: \"466388a3-4cba-4a78-a705-276c03be16fa\") " pod="cert-manager/cert-manager-55b4d954d5-vw45w"
Jul 29 18:41:35.800811 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:35.800776    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/466388a3-4cba-4a78-a705-276c03be16fa-bound-sa-token\") pod \"cert-manager-55b4d954d5-vw45w\" (UID: \"466388a3-4cba-4a78-a705-276c03be16fa\") " pod="cert-manager/cert-manager-55b4d954d5-vw45w"
Jul 29 18:41:35.800933 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:35.800822    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vzqd\" (UniqueName: \"kubernetes.io/projected/466388a3-4cba-4a78-a705-276c03be16fa-kube-api-access-6vzqd\") pod \"cert-manager-55b4d954d5-vw45w\" (UID: \"466388a3-4cba-4a78-a705-276c03be16fa\") " pod="cert-manager/cert-manager-55b4d954d5-vw45w"
Jul 29 18:41:35.893560 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:35.893443    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-vw45w"
Jul 29 18:41:36.031083 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:36.030998    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vw45w"]
Jul 29 18:41:36.033472 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:41:36.033440    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod466388a3_4cba_4a78_a705_276c03be16fa.slice/crio-62144143fb8b40334d099ab4a4181887e3e906d8ff99a135eba24441e3956dae WatchSource:0}: Error finding container 62144143fb8b40334d099ab4a4181887e3e906d8ff99a135eba24441e3956dae: Status 404 returned error can't find the container with id 62144143fb8b40334d099ab4a4181887e3e906d8ff99a135eba24441e3956dae
Jul 29 18:41:36.689436 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:36.689399    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-vw45w" event={"ID":"466388a3-4cba-4a78-a705-276c03be16fa","Type":"ContainerStarted","Data":"62144143fb8b40334d099ab4a4181887e3e906d8ff99a135eba24441e3956dae"}
Jul 29 18:41:39.708227 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:39.708183    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-vw45w" event={"ID":"466388a3-4cba-4a78-a705-276c03be16fa","Type":"ContainerStarted","Data":"5a83917e629a9f0592e9caa910859e36fd0e76262eb9395a43d96c0b038d2dad"}
Jul 29 18:41:39.727226 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:39.727170    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-vw45w" podStartSLOduration=1.6541010310000002 podStartE2EDuration="4.727150965s" totalImagesPullingTime="3.073049934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:41:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:41:39.725567281 +0000 UTC m=+425.677134360" watchObservedRunningTime="2026-07-29 18:41:39.727150965 +0000 UTC m=+425.678718041"
Jul 29 18:41:42.848270 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:42.848234    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xn9rn"]
Jul 29 18:41:42.851820 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:42.851798    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xn9rn"
Jul 29 18:41:42.859019 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:42.858993    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 29 18:41:42.859157 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:42.859062    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-6ck94\""
Jul 29 18:41:42.859157 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:42.859063    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 29 18:41:42.876059 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:42.876029    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xn9rn"]
Jul 29 18:41:42.964344 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:42.964300    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chbd2\" (UniqueName: \"kubernetes.io/projected/97fe6477-1ad4-4896-a53c-7bd8124c2e6d-kube-api-access-chbd2\") pod \"openshift-lws-operator-fd8ccff4c-xn9rn\" (UID: \"97fe6477-1ad4-4896-a53c-7bd8124c2e6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xn9rn"
Jul 29 18:41:42.964560 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:42.964366    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97fe6477-1ad4-4896-a53c-7bd8124c2e6d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xn9rn\" (UID: \"97fe6477-1ad4-4896-a53c-7bd8124c2e6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xn9rn"
Jul 29 18:41:43.065633 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:43.065583    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97fe6477-1ad4-4896-a53c-7bd8124c2e6d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xn9rn\" (UID: \"97fe6477-1ad4-4896-a53c-7bd8124c2e6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xn9rn"
Jul 29 18:41:43.065762 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:43.065693    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chbd2\" (UniqueName: \"kubernetes.io/projected/97fe6477-1ad4-4896-a53c-7bd8124c2e6d-kube-api-access-chbd2\") pod \"openshift-lws-operator-fd8ccff4c-xn9rn\" (UID: \"97fe6477-1ad4-4896-a53c-7bd8124c2e6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xn9rn"
Jul 29 18:41:43.065978 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:43.065961    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97fe6477-1ad4-4896-a53c-7bd8124c2e6d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xn9rn\" (UID: \"97fe6477-1ad4-4896-a53c-7bd8124c2e6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xn9rn"
Jul 29 18:41:43.075799 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:43.075764    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chbd2\" (UniqueName: \"kubernetes.io/projected/97fe6477-1ad4-4896-a53c-7bd8124c2e6d-kube-api-access-chbd2\") pod \"openshift-lws-operator-fd8ccff4c-xn9rn\" (UID: \"97fe6477-1ad4-4896-a53c-7bd8124c2e6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xn9rn"
Jul 29 18:41:43.161247 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:43.161156    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xn9rn"
Jul 29 18:41:43.304199 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:43.304173    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xn9rn"]
Jul 29 18:41:43.306093 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:41:43.306055    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97fe6477_1ad4_4896_a53c_7bd8124c2e6d.slice/crio-9395358407b68cccd41ce6f175c6e63e2c04db43178e8f7f9c2cd9983f8835dc WatchSource:0}: Error finding container 9395358407b68cccd41ce6f175c6e63e2c04db43178e8f7f9c2cd9983f8835dc: Status 404 returned error can't find the container with id 9395358407b68cccd41ce6f175c6e63e2c04db43178e8f7f9c2cd9983f8835dc
Jul 29 18:41:43.727965 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:43.727928    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xn9rn" event={"ID":"97fe6477-1ad4-4896-a53c-7bd8124c2e6d","Type":"ContainerStarted","Data":"9395358407b68cccd41ce6f175c6e63e2c04db43178e8f7f9c2cd9983f8835dc"}
Jul 29 18:41:46.745811 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:46.745759    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xn9rn" event={"ID":"97fe6477-1ad4-4896-a53c-7bd8124c2e6d","Type":"ContainerStarted","Data":"bc459b63e09fd0ef505a08c6c9597f5cfdd8d738ad080ed7484c9e222879fc61"}
Jul 29 18:41:46.766237 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:41:46.766180    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xn9rn" podStartSLOduration=2.214001614 podStartE2EDuration="4.766163318s" totalImagesPullingTime="2.552161704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:41:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:41:46.764492035 +0000 UTC m=+432.716059134" watchObservedRunningTime="2026-07-29 18:41:46.766163318 +0000 UTC m=+432.717730395"
Jul 29 18:42:02.745954 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.745870    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"]
Jul 29 18:42:02.749457 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.749435    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"
Jul 29 18:42:02.752455 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.752433    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 29 18:42:02.753627 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.753608    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tllmr\""
Jul 29 18:42:02.753718 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.753605    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 29 18:42:02.753718 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.753655    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 29 18:42:02.760013 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.759986    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"]
Jul 29 18:42:02.850469 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.850427    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1eef13b-9722-45d0-a109-2f07e7b3639a-metrics-cert\") pod \"lws-controller-manager-7698b7b56c-vqmtr\" (UID: \"e1eef13b-9722-45d0-a109-2f07e7b3639a\") " pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"
Jul 29 18:42:02.850469 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.850485    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69rxd\" (UniqueName: \"kubernetes.io/projected/e1eef13b-9722-45d0-a109-2f07e7b3639a-kube-api-access-69rxd\") pod \"lws-controller-manager-7698b7b56c-vqmtr\" (UID: \"e1eef13b-9722-45d0-a109-2f07e7b3639a\") " pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"
Jul 29 18:42:02.850761 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.850586    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e1eef13b-9722-45d0-a109-2f07e7b3639a-manager-config\") pod \"lws-controller-manager-7698b7b56c-vqmtr\" (UID: \"e1eef13b-9722-45d0-a109-2f07e7b3639a\") " pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"
Jul 29 18:42:02.850761 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.850682    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1eef13b-9722-45d0-a109-2f07e7b3639a-cert\") pod \"lws-controller-manager-7698b7b56c-vqmtr\" (UID: \"e1eef13b-9722-45d0-a109-2f07e7b3639a\") " pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"
Jul 29 18:42:02.951975 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.951931    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1eef13b-9722-45d0-a109-2f07e7b3639a-cert\") pod \"lws-controller-manager-7698b7b56c-vqmtr\" (UID: \"e1eef13b-9722-45d0-a109-2f07e7b3639a\") " pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"
Jul 29 18:42:02.952153 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.951989    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1eef13b-9722-45d0-a109-2f07e7b3639a-metrics-cert\") pod \"lws-controller-manager-7698b7b56c-vqmtr\" (UID: \"e1eef13b-9722-45d0-a109-2f07e7b3639a\") " pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"
Jul 29 18:42:02.952153 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.952025    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69rxd\" (UniqueName: \"kubernetes.io/projected/e1eef13b-9722-45d0-a109-2f07e7b3639a-kube-api-access-69rxd\") pod \"lws-controller-manager-7698b7b56c-vqmtr\" (UID: \"e1eef13b-9722-45d0-a109-2f07e7b3639a\") " pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"
Jul 29 18:42:02.952153 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.952065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e1eef13b-9722-45d0-a109-2f07e7b3639a-manager-config\") pod \"lws-controller-manager-7698b7b56c-vqmtr\" (UID: \"e1eef13b-9722-45d0-a109-2f07e7b3639a\") " pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"
Jul 29 18:42:02.952776 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.952752    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e1eef13b-9722-45d0-a109-2f07e7b3639a-manager-config\") pod \"lws-controller-manager-7698b7b56c-vqmtr\" (UID: \"e1eef13b-9722-45d0-a109-2f07e7b3639a\") " pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"
Jul 29 18:42:02.954444 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.954422    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1eef13b-9722-45d0-a109-2f07e7b3639a-cert\") pod \"lws-controller-manager-7698b7b56c-vqmtr\" (UID: \"e1eef13b-9722-45d0-a109-2f07e7b3639a\") " pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"
Jul 29 18:42:02.954554 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.954460    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1eef13b-9722-45d0-a109-2f07e7b3639a-metrics-cert\") pod \"lws-controller-manager-7698b7b56c-vqmtr\" (UID: \"e1eef13b-9722-45d0-a109-2f07e7b3639a\") " pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"
Jul 29 18:42:02.967337 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:02.967300    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69rxd\" (UniqueName: \"kubernetes.io/projected/e1eef13b-9722-45d0-a109-2f07e7b3639a-kube-api-access-69rxd\") pod \"lws-controller-manager-7698b7b56c-vqmtr\" (UID: \"e1eef13b-9722-45d0-a109-2f07e7b3639a\") " pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"
Jul 29 18:42:03.059481 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:03.059438    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"
Jul 29 18:42:03.202458 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:03.202358    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"]
Jul 29 18:42:03.204681 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:42:03.204655    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1eef13b_9722_45d0_a109_2f07e7b3639a.slice/crio-76e9ffce020e36e5e20bf0303f2f8b38ca72e60ba651e8cffcad29a2c33c9cd6 WatchSource:0}: Error finding container 76e9ffce020e36e5e20bf0303f2f8b38ca72e60ba651e8cffcad29a2c33c9cd6: Status 404 returned error can't find the container with id 76e9ffce020e36e5e20bf0303f2f8b38ca72e60ba651e8cffcad29a2c33c9cd6
Jul 29 18:42:03.824625 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:03.824590    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr" event={"ID":"e1eef13b-9722-45d0-a109-2f07e7b3639a","Type":"ContainerStarted","Data":"76e9ffce020e36e5e20bf0303f2f8b38ca72e60ba651e8cffcad29a2c33c9cd6"}
Jul 29 18:42:04.831317 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:04.831283    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr" event={"ID":"e1eef13b-9722-45d0-a109-2f07e7b3639a","Type":"ContainerStarted","Data":"94dbed12b3b12f334e1fbf29a2096fb4fb2131321af68ba9dc5099f2e7822454"}
Jul 29 18:42:04.831752 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:04.831397    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"
Jul 29 18:42:04.850043 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:04.849989    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr" podStartSLOduration=1.380698356 podStartE2EDuration="2.849972814s" totalImagesPullingTime="1.469274458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:42:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:42:04.848706513 +0000 UTC m=+450.800273610" watchObservedRunningTime="2026-07-29 18:42:04.849972814 +0000 UTC m=+450.801539891"
Jul 29 18:42:15.839352 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:15.839321    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7698b7b56c-vqmtr"
Jul 29 18:42:36.419794 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:36.419756    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8dc7bc744-x87d5"]
Jul 29 18:42:42.276342 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.276289    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv"]
Jul 29 18:42:42.279192 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.279175    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv"
Jul 29 18:42:42.281966 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.281935    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 18:42:42.282116 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.281989    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 18:42:42.282116 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.282088    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2dlbv\""
Jul 29 18:42:42.283097 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.283076    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 29 18:42:42.283097 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.283086    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 29 18:42:42.287933 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.287910    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv"]
Jul 29 18:42:42.400628 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.400586    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkr4x\" (UniqueName: \"kubernetes.io/projected/12431451-5505-490c-8fa0-ae5aa5e2c7dc-kube-api-access-nkr4x\") pod \"kuadrant-console-plugin-598f86468b-r27zv\" (UID: \"12431451-5505-490c-8fa0-ae5aa5e2c7dc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv"
Jul 29 18:42:42.400628 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.400634    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/12431451-5505-490c-8fa0-ae5aa5e2c7dc-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-r27zv\" (UID: \"12431451-5505-490c-8fa0-ae5aa5e2c7dc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv"
Jul 29 18:42:42.400844 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.400716    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/12431451-5505-490c-8fa0-ae5aa5e2c7dc-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-r27zv\" (UID: \"12431451-5505-490c-8fa0-ae5aa5e2c7dc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv"
Jul 29 18:42:42.502244 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.502205    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/12431451-5505-490c-8fa0-ae5aa5e2c7dc-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-r27zv\" (UID: \"12431451-5505-490c-8fa0-ae5aa5e2c7dc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv"
Jul 29 18:42:42.502438 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.502291    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/12431451-5505-490c-8fa0-ae5aa5e2c7dc-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-r27zv\" (UID: \"12431451-5505-490c-8fa0-ae5aa5e2c7dc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv"
Jul 29 18:42:42.502438 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.502428    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkr4x\" (UniqueName: \"kubernetes.io/projected/12431451-5505-490c-8fa0-ae5aa5e2c7dc-kube-api-access-nkr4x\") pod \"kuadrant-console-plugin-598f86468b-r27zv\" (UID: \"12431451-5505-490c-8fa0-ae5aa5e2c7dc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv"
Jul 29 18:42:42.503058 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.503028    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/12431451-5505-490c-8fa0-ae5aa5e2c7dc-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-r27zv\" (UID: \"12431451-5505-490c-8fa0-ae5aa5e2c7dc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv"
Jul 29 18:42:42.504788 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.504763    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/12431451-5505-490c-8fa0-ae5aa5e2c7dc-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-r27zv\" (UID: \"12431451-5505-490c-8fa0-ae5aa5e2c7dc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv"
Jul 29 18:42:42.510962 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.510934    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkr4x\" (UniqueName: \"kubernetes.io/projected/12431451-5505-490c-8fa0-ae5aa5e2c7dc-kube-api-access-nkr4x\") pod \"kuadrant-console-plugin-598f86468b-r27zv\" (UID: \"12431451-5505-490c-8fa0-ae5aa5e2c7dc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv"
Jul 29 18:42:42.589816 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.589720    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv"
Jul 29 18:42:42.732101 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:42.731925    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv"]
Jul 29 18:42:42.734871 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:42:42.734840    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12431451_5505_490c_8fa0_ae5aa5e2c7dc.slice/crio-c20fccbc40effb12472846ec080065ca1ef1865a3c8404f56980b0628e0f3060 WatchSource:0}: Error finding container c20fccbc40effb12472846ec080065ca1ef1865a3c8404f56980b0628e0f3060: Status 404 returned error can't find the container with id c20fccbc40effb12472846ec080065ca1ef1865a3c8404f56980b0628e0f3060
Jul 29 18:42:43.016834 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:43.016799    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv" event={"ID":"12431451-5505-490c-8fa0-ae5aa5e2c7dc","Type":"ContainerStarted","Data":"c20fccbc40effb12472846ec080065ca1ef1865a3c8404f56980b0628e0f3060"}
Jul 29 18:42:49.051987 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:49.051950    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv" event={"ID":"12431451-5505-490c-8fa0-ae5aa5e2c7dc","Type":"ContainerStarted","Data":"e47260ae4e1fd0814cdd2183140d7d4bd2b47be749336a3a3412b1eb2830eefa"}
Jul 29 18:42:49.069484 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:42:49.069425    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r27zv" podStartSLOduration=1.637367759 podStartE2EDuration="7.069408602s" totalImagesPullingTime="5.432040843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:42:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:42:49.068009611 +0000 UTC m=+495.019576713" watchObservedRunningTime="2026-07-29 18:42:49.069408602 +0000 UTC m=+495.020975697"
Jul 29 18:43:01.440459 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.440395    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8dc7bc744-x87d5" podUID="2359f584-67d9-4b8f-a2e4-86af703088fd" containerName="console" containerID="cri-o://69cf5323bb8eaa8501e823c6fb12671fac29813e8086c7c8a38141bfeb24ed62" gracePeriod=15
Jul 29 18:43:01.685944 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.685916    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:43:01.777013 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.776978    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92pd4\" (UniqueName: \"kubernetes.io/projected/2359f584-67d9-4b8f-a2e4-86af703088fd-kube-api-access-92pd4\") pod \"2359f584-67d9-4b8f-a2e4-86af703088fd\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") "
Jul 29 18:43:01.777223 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.777024    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2359f584-67d9-4b8f-a2e4-86af703088fd-console-oauth-config\") pod \"2359f584-67d9-4b8f-a2e4-86af703088fd\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") "
Jul 29 18:43:01.777223 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.777057    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2359f584-67d9-4b8f-a2e4-86af703088fd-console-serving-cert\") pod \"2359f584-67d9-4b8f-a2e4-86af703088fd\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") "
Jul 29 18:43:01.777223 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.777095    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-oauth-serving-cert\") pod \"2359f584-67d9-4b8f-a2e4-86af703088fd\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") "
Jul 29 18:43:01.777223 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.777148    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-trusted-ca-bundle\") pod \"2359f584-67d9-4b8f-a2e4-86af703088fd\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") "
Jul 29 18:43:01.777223 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.777188    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-service-ca\") pod \"2359f584-67d9-4b8f-a2e4-86af703088fd\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") "
Jul 29 18:43:01.777223 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.777220    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-console-config\") pod \"2359f584-67d9-4b8f-a2e4-86af703088fd\" (UID: \"2359f584-67d9-4b8f-a2e4-86af703088fd\") "
Jul 29 18:43:01.777788 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.777706    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2359f584-67d9-4b8f-a2e4-86af703088fd" (UID: "2359f584-67d9-4b8f-a2e4-86af703088fd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:43:01.777895 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.777844    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-service-ca" (OuterVolumeSpecName: "service-ca") pod "2359f584-67d9-4b8f-a2e4-86af703088fd" (UID: "2359f584-67d9-4b8f-a2e4-86af703088fd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:43:01.777947 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.777899    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-console-config" (OuterVolumeSpecName: "console-config") pod "2359f584-67d9-4b8f-a2e4-86af703088fd" (UID: "2359f584-67d9-4b8f-a2e4-86af703088fd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:43:01.778076 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.778055    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2359f584-67d9-4b8f-a2e4-86af703088fd" (UID: "2359f584-67d9-4b8f-a2e4-86af703088fd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:43:01.779440 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.779406    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2359f584-67d9-4b8f-a2e4-86af703088fd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2359f584-67d9-4b8f-a2e4-86af703088fd" (UID: "2359f584-67d9-4b8f-a2e4-86af703088fd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:43:01.779691 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.779671    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2359f584-67d9-4b8f-a2e4-86af703088fd-kube-api-access-92pd4" (OuterVolumeSpecName: "kube-api-access-92pd4") pod "2359f584-67d9-4b8f-a2e4-86af703088fd" (UID: "2359f584-67d9-4b8f-a2e4-86af703088fd"). InnerVolumeSpecName "kube-api-access-92pd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:43:01.779771 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.779719    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2359f584-67d9-4b8f-a2e4-86af703088fd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2359f584-67d9-4b8f-a2e4-86af703088fd" (UID: "2359f584-67d9-4b8f-a2e4-86af703088fd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:43:01.878321 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.878279    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92pd4\" (UniqueName: \"kubernetes.io/projected/2359f584-67d9-4b8f-a2e4-86af703088fd-kube-api-access-92pd4\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:43:01.878321 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.878314    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2359f584-67d9-4b8f-a2e4-86af703088fd-console-oauth-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:43:01.878321 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.878324    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2359f584-67d9-4b8f-a2e4-86af703088fd-console-serving-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:43:01.878598 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.878333    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-oauth-serving-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:43:01.878598 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.878342    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-trusted-ca-bundle\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:43:01.878598 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.878351    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-service-ca\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:43:01.878598 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:01.878360    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2359f584-67d9-4b8f-a2e4-86af703088fd-console-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:43:02.114116 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:02.114016    2579 generic.go:356] "Generic (PLEG): container finished" podID="2359f584-67d9-4b8f-a2e4-86af703088fd" containerID="69cf5323bb8eaa8501e823c6fb12671fac29813e8086c7c8a38141bfeb24ed62" exitCode=0
Jul 29 18:43:02.114116 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:02.114075    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8dc7bc744-x87d5"
Jul 29 18:43:02.114296 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:02.114125    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8dc7bc744-x87d5" event={"ID":"2359f584-67d9-4b8f-a2e4-86af703088fd","Type":"ContainerDied","Data":"69cf5323bb8eaa8501e823c6fb12671fac29813e8086c7c8a38141bfeb24ed62"}
Jul 29 18:43:02.114296 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:02.114212    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8dc7bc744-x87d5" event={"ID":"2359f584-67d9-4b8f-a2e4-86af703088fd","Type":"ContainerDied","Data":"92c475e7cda7305f1bd9c1f57ea1347f93e611940bddee8fc6c65d6c3a350519"}
Jul 29 18:43:02.114296 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:02.114239    2579 scope.go:117] "RemoveContainer" containerID="69cf5323bb8eaa8501e823c6fb12671fac29813e8086c7c8a38141bfeb24ed62"
Jul 29 18:43:02.123981 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:02.123961    2579 scope.go:117] "RemoveContainer" containerID="69cf5323bb8eaa8501e823c6fb12671fac29813e8086c7c8a38141bfeb24ed62"
Jul 29 18:43:02.124332 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:43:02.124312    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69cf5323bb8eaa8501e823c6fb12671fac29813e8086c7c8a38141bfeb24ed62\": container with ID starting with 69cf5323bb8eaa8501e823c6fb12671fac29813e8086c7c8a38141bfeb24ed62 not found: ID does not exist" containerID="69cf5323bb8eaa8501e823c6fb12671fac29813e8086c7c8a38141bfeb24ed62"
Jul 29 18:43:02.124388 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:02.124342    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69cf5323bb8eaa8501e823c6fb12671fac29813e8086c7c8a38141bfeb24ed62"} err="failed to get container status \"69cf5323bb8eaa8501e823c6fb12671fac29813e8086c7c8a38141bfeb24ed62\": rpc error: code = NotFound desc = could not find container \"69cf5323bb8eaa8501e823c6fb12671fac29813e8086c7c8a38141bfeb24ed62\": container with ID starting with 69cf5323bb8eaa8501e823c6fb12671fac29813e8086c7c8a38141bfeb24ed62 not found: ID does not exist"
Jul 29 18:43:02.138817 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:02.138774    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8dc7bc744-x87d5"]
Jul 29 18:43:02.142303 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:02.142274    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8dc7bc744-x87d5"]
Jul 29 18:43:02.585650 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:02.585611    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2359f584-67d9-4b8f-a2e4-86af703088fd" path="/var/lib/kubelet/pods/2359f584-67d9-4b8f-a2e4-86af703088fd/volumes"
Jul 29 18:43:24.709009 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:24.708973    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-xrkqh"]
Jul 29 18:43:24.709497 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:24.709347    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2359f584-67d9-4b8f-a2e4-86af703088fd" containerName="console"
Jul 29 18:43:24.709497 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:24.709359    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2359f584-67d9-4b8f-a2e4-86af703088fd" containerName="console"
Jul 29 18:43:24.709497 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:24.709423    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2359f584-67d9-4b8f-a2e4-86af703088fd" containerName="console"
Jul 29 18:43:24.730981 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:24.730950    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-xrkqh"]
Jul 29 18:43:24.731130 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:24.731043    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-xrkqh"
Jul 29 18:43:24.733649 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:24.733625    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fzz9w\""
Jul 29 18:43:24.824035 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:24.824004    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-85z7t"]
Jul 29 18:43:24.827959 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:24.827936    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-85z7t"
Jul 29 18:43:24.834038 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:24.834002    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-85z7t"]
Jul 29 18:43:24.888268 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:24.888228    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v4dt\" (UniqueName: \"kubernetes.io/projected/2b729f27-a15a-4640-a4ca-8dbc8a7317b7-kube-api-access-7v4dt\") pod \"authorino-b54df94b4-xrkqh\" (UID: \"2b729f27-a15a-4640-a4ca-8dbc8a7317b7\") " pod="kuadrant-system/authorino-b54df94b4-xrkqh"
Jul 29 18:43:24.989380 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:24.989282    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4dt\" (UniqueName: \"kubernetes.io/projected/2b729f27-a15a-4640-a4ca-8dbc8a7317b7-kube-api-access-7v4dt\") pod \"authorino-b54df94b4-xrkqh\" (UID: \"2b729f27-a15a-4640-a4ca-8dbc8a7317b7\") " pod="kuadrant-system/authorino-b54df94b4-xrkqh"
Jul 29 18:43:24.989596 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:24.989423    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfcmk\" (UniqueName: \"kubernetes.io/projected/2d877cbb-14c7-4cd2-a764-642daa788bf6-kube-api-access-bfcmk\") pod \"authorino-c548b5d7d-85z7t\" (UID: \"2d877cbb-14c7-4cd2-a764-642daa788bf6\") " pod="kuadrant-system/authorino-c548b5d7d-85z7t"
Jul 29 18:43:24.998460 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:24.998439    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v4dt\" (UniqueName: \"kubernetes.io/projected/2b729f27-a15a-4640-a4ca-8dbc8a7317b7-kube-api-access-7v4dt\") pod \"authorino-b54df94b4-xrkqh\" (UID: \"2b729f27-a15a-4640-a4ca-8dbc8a7317b7\") " pod="kuadrant-system/authorino-b54df94b4-xrkqh"
Jul 29 18:43:25.042701 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:25.042652    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-xrkqh"
Jul 29 18:43:25.090082 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:25.090038    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfcmk\" (UniqueName: \"kubernetes.io/projected/2d877cbb-14c7-4cd2-a764-642daa788bf6-kube-api-access-bfcmk\") pod \"authorino-c548b5d7d-85z7t\" (UID: \"2d877cbb-14c7-4cd2-a764-642daa788bf6\") " pod="kuadrant-system/authorino-c548b5d7d-85z7t"
Jul 29 18:43:25.099578 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:25.099547    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfcmk\" (UniqueName: \"kubernetes.io/projected/2d877cbb-14c7-4cd2-a764-642daa788bf6-kube-api-access-bfcmk\") pod \"authorino-c548b5d7d-85z7t\" (UID: \"2d877cbb-14c7-4cd2-a764-642daa788bf6\") " pod="kuadrant-system/authorino-c548b5d7d-85z7t"
Jul 29 18:43:25.138840 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:25.138805    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-85z7t"
Jul 29 18:43:25.187453 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:25.187421    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-xrkqh"]
Jul 29 18:43:25.191304 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:43:25.191264    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b729f27_a15a_4640_a4ca_8dbc8a7317b7.slice/crio-7859cf068df50167fd18543db7709725a74c3d4dec34f0468b4350b8b2dd5ea4 WatchSource:0}: Error finding container 7859cf068df50167fd18543db7709725a74c3d4dec34f0468b4350b8b2dd5ea4: Status 404 returned error can't find the container with id 7859cf068df50167fd18543db7709725a74c3d4dec34f0468b4350b8b2dd5ea4
Jul 29 18:43:25.228818 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:25.228776    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-xrkqh" event={"ID":"2b729f27-a15a-4640-a4ca-8dbc8a7317b7","Type":"ContainerStarted","Data":"7859cf068df50167fd18543db7709725a74c3d4dec34f0468b4350b8b2dd5ea4"}
Jul 29 18:43:25.288309 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:25.288281    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-85z7t"]
Jul 29 18:43:25.291055 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:43:25.291012    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d877cbb_14c7_4cd2_a764_642daa788bf6.slice/crio-bea1d802216455ce0a258c0341f96ed24d9b2d98163a67034b004dd7b58cf84f WatchSource:0}: Error finding container bea1d802216455ce0a258c0341f96ed24d9b2d98163a67034b004dd7b58cf84f: Status 404 returned error can't find the container with id bea1d802216455ce0a258c0341f96ed24d9b2d98163a67034b004dd7b58cf84f
Jul 29 18:43:26.235383 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:26.235342    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-85z7t" event={"ID":"2d877cbb-14c7-4cd2-a764-642daa788bf6","Type":"ContainerStarted","Data":"bea1d802216455ce0a258c0341f96ed24d9b2d98163a67034b004dd7b58cf84f"}
Jul 29 18:43:29.253671 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:29.253630    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-85z7t" event={"ID":"2d877cbb-14c7-4cd2-a764-642daa788bf6","Type":"ContainerStarted","Data":"8c26501c4557bed4a493d14d08add1ad25424aafb2598731322f78aa08830bf4"}
Jul 29 18:43:29.270134 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:29.270056    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-85z7t" podStartSLOduration=2.130178757 podStartE2EDuration="5.270038959s" totalImagesPullingTime="3.139860202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:43:29.269209111 +0000 UTC m=+535.220776201" watchObservedRunningTime="2026-07-29 18:43:29.270038959 +0000 UTC m=+535.221606036"
Jul 29 18:43:29.292448 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:29.292419    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-xrkqh"]
Jul 29 18:43:36.290614 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:36.290578    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-xrkqh" event={"ID":"2b729f27-a15a-4640-a4ca-8dbc8a7317b7","Type":"ContainerStarted","Data":"0a73aac10a30b5a7d16f19790b2683b41c65c29d9fe611dc445d2baad353b28b"}
Jul 29 18:43:36.291119 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:36.290651    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-xrkqh" podUID="2b729f27-a15a-4640-a4ca-8dbc8a7317b7" containerName="authorino" containerID="cri-o://0a73aac10a30b5a7d16f19790b2683b41c65c29d9fe611dc445d2baad353b28b" gracePeriod=30
Jul 29 18:43:36.307088 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:36.306939    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-xrkqh" podStartSLOduration=1.3088618890000001 podStartE2EDuration="12.306921541s" totalImagesPullingTime="10.998059652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:43:36.305625923 +0000 UTC m=+542.257193014" watchObservedRunningTime="2026-07-29 18:43:36.306921541 +0000 UTC m=+542.258488656"
Jul 29 18:43:36.534081 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:36.534044    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-xrkqh"
Jul 29 18:43:36.699655 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:36.699610    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7v4dt\" (UniqueName: \"kubernetes.io/projected/2b729f27-a15a-4640-a4ca-8dbc8a7317b7-kube-api-access-7v4dt\") pod \"2b729f27-a15a-4640-a4ca-8dbc8a7317b7\" (UID: \"2b729f27-a15a-4640-a4ca-8dbc8a7317b7\") "
Jul 29 18:43:36.701852 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:36.701817    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b729f27-a15a-4640-a4ca-8dbc8a7317b7-kube-api-access-7v4dt" (OuterVolumeSpecName: "kube-api-access-7v4dt") pod "2b729f27-a15a-4640-a4ca-8dbc8a7317b7" (UID: "2b729f27-a15a-4640-a4ca-8dbc8a7317b7"). InnerVolumeSpecName "kube-api-access-7v4dt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:43:36.800460 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:36.800413    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7v4dt\" (UniqueName: \"kubernetes.io/projected/2b729f27-a15a-4640-a4ca-8dbc8a7317b7-kube-api-access-7v4dt\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:43:37.297011 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:37.296978    2579 generic.go:356] "Generic (PLEG): container finished" podID="2b729f27-a15a-4640-a4ca-8dbc8a7317b7" containerID="0a73aac10a30b5a7d16f19790b2683b41c65c29d9fe611dc445d2baad353b28b" exitCode=0
Jul 29 18:43:37.297442 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:37.297037    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-xrkqh"
Jul 29 18:43:37.297442 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:37.297064    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-xrkqh" event={"ID":"2b729f27-a15a-4640-a4ca-8dbc8a7317b7","Type":"ContainerDied","Data":"0a73aac10a30b5a7d16f19790b2683b41c65c29d9fe611dc445d2baad353b28b"}
Jul 29 18:43:37.297442 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:37.297110    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-xrkqh" event={"ID":"2b729f27-a15a-4640-a4ca-8dbc8a7317b7","Type":"ContainerDied","Data":"7859cf068df50167fd18543db7709725a74c3d4dec34f0468b4350b8b2dd5ea4"}
Jul 29 18:43:37.297442 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:37.297132    2579 scope.go:117] "RemoveContainer" containerID="0a73aac10a30b5a7d16f19790b2683b41c65c29d9fe611dc445d2baad353b28b"
Jul 29 18:43:37.305850 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:37.305828    2579 scope.go:117] "RemoveContainer" containerID="0a73aac10a30b5a7d16f19790b2683b41c65c29d9fe611dc445d2baad353b28b"
Jul 29 18:43:37.306178 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:43:37.306158    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a73aac10a30b5a7d16f19790b2683b41c65c29d9fe611dc445d2baad353b28b\": container with ID starting with 0a73aac10a30b5a7d16f19790b2683b41c65c29d9fe611dc445d2baad353b28b not found: ID does not exist" containerID="0a73aac10a30b5a7d16f19790b2683b41c65c29d9fe611dc445d2baad353b28b"
Jul 29 18:43:37.306230 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:37.306188    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a73aac10a30b5a7d16f19790b2683b41c65c29d9fe611dc445d2baad353b28b"} err="failed to get container status \"0a73aac10a30b5a7d16f19790b2683b41c65c29d9fe611dc445d2baad353b28b\": rpc error: code = NotFound desc = could not find container \"0a73aac10a30b5a7d16f19790b2683b41c65c29d9fe611dc445d2baad353b28b\": container with ID starting with 0a73aac10a30b5a7d16f19790b2683b41c65c29d9fe611dc445d2baad353b28b not found: ID does not exist"
Jul 29 18:43:37.320977 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:37.320946    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-xrkqh"]
Jul 29 18:43:37.325356 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:37.325328    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-xrkqh"]
Jul 29 18:43:38.585778 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:38.585743    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b729f27-a15a-4640-a4ca-8dbc8a7317b7" path="/var/lib/kubelet/pods/2b729f27-a15a-4640-a4ca-8dbc8a7317b7/volumes"
Jul 29 18:43:47.337719 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:47.337681    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-867c7768fd-69w2s"]
Jul 29 18:43:47.338116 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:47.338101    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b729f27-a15a-4640-a4ca-8dbc8a7317b7" containerName="authorino"
Jul 29 18:43:47.338178 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:47.338117    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b729f27-a15a-4640-a4ca-8dbc8a7317b7" containerName="authorino"
Jul 29 18:43:47.338218 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:47.338201    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b729f27-a15a-4640-a4ca-8dbc8a7317b7" containerName="authorino"
Jul 29 18:43:47.341388 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:47.341371    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-867c7768fd-69w2s"
Jul 29 18:43:47.344208 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:47.344184    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 29 18:43:47.352048 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:47.352024    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-867c7768fd-69w2s"]
Jul 29 18:43:47.391032 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:47.391000    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rrbr\" (UniqueName: \"kubernetes.io/projected/041bfccc-5d61-41f9-9062-891c1d136c09-kube-api-access-9rrbr\") pod \"authorino-867c7768fd-69w2s\" (UID: \"041bfccc-5d61-41f9-9062-891c1d136c09\") " pod="kuadrant-system/authorino-867c7768fd-69w2s"
Jul 29 18:43:47.391215 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:47.391063    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/041bfccc-5d61-41f9-9062-891c1d136c09-tls-cert\") pod \"authorino-867c7768fd-69w2s\" (UID: \"041bfccc-5d61-41f9-9062-891c1d136c09\") " pod="kuadrant-system/authorino-867c7768fd-69w2s"
Jul 29 18:43:47.491680 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:47.491640    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rrbr\" (UniqueName: \"kubernetes.io/projected/041bfccc-5d61-41f9-9062-891c1d136c09-kube-api-access-9rrbr\") pod \"authorino-867c7768fd-69w2s\" (UID: \"041bfccc-5d61-41f9-9062-891c1d136c09\") " pod="kuadrant-system/authorino-867c7768fd-69w2s"
Jul 29 18:43:47.491857 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:47.491705    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/041bfccc-5d61-41f9-9062-891c1d136c09-tls-cert\") pod \"authorino-867c7768fd-69w2s\" (UID: \"041bfccc-5d61-41f9-9062-891c1d136c09\") " pod="kuadrant-system/authorino-867c7768fd-69w2s"
Jul 29 18:43:47.494140 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:47.494110    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/041bfccc-5d61-41f9-9062-891c1d136c09-tls-cert\") pod \"authorino-867c7768fd-69w2s\" (UID: \"041bfccc-5d61-41f9-9062-891c1d136c09\") " pod="kuadrant-system/authorino-867c7768fd-69w2s"
Jul 29 18:43:47.500975 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:47.500953    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rrbr\" (UniqueName: \"kubernetes.io/projected/041bfccc-5d61-41f9-9062-891c1d136c09-kube-api-access-9rrbr\") pod \"authorino-867c7768fd-69w2s\" (UID: \"041bfccc-5d61-41f9-9062-891c1d136c09\") " pod="kuadrant-system/authorino-867c7768fd-69w2s"
Jul 29 18:43:47.651766 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:47.651675    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-867c7768fd-69w2s"
Jul 29 18:43:47.794635 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:47.794610    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-867c7768fd-69w2s"]
Jul 29 18:43:47.797256 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:43:47.797220    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod041bfccc_5d61_41f9_9062_891c1d136c09.slice/crio-1ff5f7164a3206e3c44e5c5ddce3a34c991f0bf5568b58aae508339147e3c81d WatchSource:0}: Error finding container 1ff5f7164a3206e3c44e5c5ddce3a34c991f0bf5568b58aae508339147e3c81d: Status 404 returned error can't find the container with id 1ff5f7164a3206e3c44e5c5ddce3a34c991f0bf5568b58aae508339147e3c81d
Jul 29 18:43:48.352093 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:48.352057    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-867c7768fd-69w2s" event={"ID":"041bfccc-5d61-41f9-9062-891c1d136c09","Type":"ContainerStarted","Data":"1ff5f7164a3206e3c44e5c5ddce3a34c991f0bf5568b58aae508339147e3c81d"}
Jul 29 18:43:49.358556 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:49.358495    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-867c7768fd-69w2s" event={"ID":"041bfccc-5d61-41f9-9062-891c1d136c09","Type":"ContainerStarted","Data":"20536b86c3e5a5aaaa005814444850987a19e46a14886d4afdda22cd556fe66d"}
Jul 29 18:43:49.378043 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:49.377990    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-867c7768fd-69w2s" podStartSLOduration=1.441737564 podStartE2EDuration="2.377969165s" totalImagesPullingTime="936.231601ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:43:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:43:49.37787965 +0000 UTC m=+555.329446728" watchObservedRunningTime="2026-07-29 18:43:49.377969165 +0000 UTC m=+555.329536242"
Jul 29 18:43:49.407626 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:49.407582    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-85z7t"]
Jul 29 18:43:49.407812 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:49.407776    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-85z7t" podUID="2d877cbb-14c7-4cd2-a764-642daa788bf6" containerName="authorino" containerID="cri-o://8c26501c4557bed4a493d14d08add1ad25424aafb2598731322f78aa08830bf4" gracePeriod=30
Jul 29 18:43:49.664708 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:49.664679    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-85z7t"
Jul 29 18:43:49.711655 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:49.711621    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfcmk\" (UniqueName: \"kubernetes.io/projected/2d877cbb-14c7-4cd2-a764-642daa788bf6-kube-api-access-bfcmk\") pod \"2d877cbb-14c7-4cd2-a764-642daa788bf6\" (UID: \"2d877cbb-14c7-4cd2-a764-642daa788bf6\") "
Jul 29 18:43:49.713766 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:49.713736    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d877cbb-14c7-4cd2-a764-642daa788bf6-kube-api-access-bfcmk" (OuterVolumeSpecName: "kube-api-access-bfcmk") pod "2d877cbb-14c7-4cd2-a764-642daa788bf6" (UID: "2d877cbb-14c7-4cd2-a764-642daa788bf6"). InnerVolumeSpecName "kube-api-access-bfcmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:43:49.812546 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:49.812480    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfcmk\" (UniqueName: \"kubernetes.io/projected/2d877cbb-14c7-4cd2-a764-642daa788bf6-kube-api-access-bfcmk\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:43:50.364895 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:50.364853    2579 generic.go:356] "Generic (PLEG): container finished" podID="2d877cbb-14c7-4cd2-a764-642daa788bf6" containerID="8c26501c4557bed4a493d14d08add1ad25424aafb2598731322f78aa08830bf4" exitCode=0
Jul 29 18:43:50.364895 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:50.364901    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-85z7t"
Jul 29 18:43:50.365352 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:50.364937    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-85z7t" event={"ID":"2d877cbb-14c7-4cd2-a764-642daa788bf6","Type":"ContainerDied","Data":"8c26501c4557bed4a493d14d08add1ad25424aafb2598731322f78aa08830bf4"}
Jul 29 18:43:50.365352 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:50.364977    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-85z7t" event={"ID":"2d877cbb-14c7-4cd2-a764-642daa788bf6","Type":"ContainerDied","Data":"bea1d802216455ce0a258c0341f96ed24d9b2d98163a67034b004dd7b58cf84f"}
Jul 29 18:43:50.365352 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:50.364994    2579 scope.go:117] "RemoveContainer" containerID="8c26501c4557bed4a493d14d08add1ad25424aafb2598731322f78aa08830bf4"
Jul 29 18:43:50.373894 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:50.373874    2579 scope.go:117] "RemoveContainer" containerID="8c26501c4557bed4a493d14d08add1ad25424aafb2598731322f78aa08830bf4"
Jul 29 18:43:50.374220 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:43:50.374200    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c26501c4557bed4a493d14d08add1ad25424aafb2598731322f78aa08830bf4\": container with ID starting with 8c26501c4557bed4a493d14d08add1ad25424aafb2598731322f78aa08830bf4 not found: ID does not exist" containerID="8c26501c4557bed4a493d14d08add1ad25424aafb2598731322f78aa08830bf4"
Jul 29 18:43:50.374305 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:50.374229    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c26501c4557bed4a493d14d08add1ad25424aafb2598731322f78aa08830bf4"} err="failed to get container status \"8c26501c4557bed4a493d14d08add1ad25424aafb2598731322f78aa08830bf4\": rpc error: code = NotFound desc = could not find container \"8c26501c4557bed4a493d14d08add1ad25424aafb2598731322f78aa08830bf4\": container with ID starting with 8c26501c4557bed4a493d14d08add1ad25424aafb2598731322f78aa08830bf4 not found: ID does not exist"
Jul 29 18:43:50.386692 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:50.386658    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-85z7t"]
Jul 29 18:43:50.393791 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:50.393762    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-85z7t"]
Jul 29 18:43:50.585934 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:43:50.585897    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d877cbb-14c7-4cd2-a764-642daa788bf6" path="/var/lib/kubelet/pods/2d877cbb-14c7-4cd2-a764-642daa788bf6/volumes"
Jul 29 18:44:34.442880 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:44:34.442851    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 18:44:34.444476 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:44:34.444452    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 18:46:03.847946 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.847908    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"]
Jul 29 18:46:03.848495 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.848277    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d877cbb-14c7-4cd2-a764-642daa788bf6" containerName="authorino"
Jul 29 18:46:03.848495 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.848288    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d877cbb-14c7-4cd2-a764-642daa788bf6" containerName="authorino"
Jul 29 18:46:03.848495 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.848383    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d877cbb-14c7-4cd2-a764-642daa788bf6" containerName="authorino"
Jul 29 18:46:03.850662 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.850640    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:03.854662 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.854635    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2cmt9\""
Jul 29 18:46:03.854817 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.854705    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
Jul 29 18:46:03.854817 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.854786    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 29 18:46:03.854927 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.854877    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 29 18:46:03.861675 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.861640    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"]
Jul 29 18:46:03.918852 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.918816    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-model-cache\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:03.918852 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.918858    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ggmt\" (UniqueName: \"kubernetes.io/projected/eff69ae2-ba13-4f60-99f6-106af37b8e1c-kube-api-access-4ggmt\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:03.919083 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.918898    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-home\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:03.919083 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.918936    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-tmp-dir\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:03.919083 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.918958    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-dshm\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:03.919083 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.918976    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:03.919083 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:03.919014    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eff69ae2-ba13-4f60-99f6-106af37b8e1c-tls-certs\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:04.020018 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.019974    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-home\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:04.020018 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.020021    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-tmp-dir\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:04.020284 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.020041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-dshm\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:04.020284 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.020061    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:04.020284 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.020193    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eff69ae2-ba13-4f60-99f6-106af37b8e1c-tls-certs\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:04.020439 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.020337    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-model-cache\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:04.020439 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.020368    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ggmt\" (UniqueName: \"kubernetes.io/projected/eff69ae2-ba13-4f60-99f6-106af37b8e1c-kube-api-access-4ggmt\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:04.020439 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.020423    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-home\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:04.020680 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.020469    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-tmp-dir\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:04.020680 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.020608    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:04.020792 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.020687    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-model-cache\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:04.022566 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.022539    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-dshm\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:04.022772 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.022758    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eff69ae2-ba13-4f60-99f6-106af37b8e1c-tls-certs\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:04.029528 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.029492    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ggmt\" (UniqueName: \"kubernetes.io/projected/eff69ae2-ba13-4f60-99f6-106af37b8e1c-kube-api-access-4ggmt\") pod \"scheduler-inline-config-test-kserve-5db4fdc874-zfj72\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:04.162546 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.162422    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:04.310366 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.310341    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"]
Jul 29 18:46:04.312591 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:46:04.312559    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff69ae2_ba13_4f60_99f6_106af37b8e1c.slice/crio-001cf023c31998135cad0597f48b61295d92cb7da28f9bf8c64f094c2316656c WatchSource:0}: Error finding container 001cf023c31998135cad0597f48b61295d92cb7da28f9bf8c64f094c2316656c: Status 404 returned error can't find the container with id 001cf023c31998135cad0597f48b61295d92cb7da28f9bf8c64f094c2316656c
Jul 29 18:46:04.314658 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:04.314639    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:46:05.018493 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:05.018455    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72" event={"ID":"eff69ae2-ba13-4f60-99f6-106af37b8e1c","Type":"ContainerStarted","Data":"001cf023c31998135cad0597f48b61295d92cb7da28f9bf8c64f094c2316656c"}
Jul 29 18:46:09.043458 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:09.043417    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72" event={"ID":"eff69ae2-ba13-4f60-99f6-106af37b8e1c","Type":"ContainerStarted","Data":"8bf1db9505c3781de755a80768ff86c6d954dc23d3281cce7b5b0066565b482c"}
Jul 29 18:46:14.069901 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:14.069866    2579 generic.go:356] "Generic (PLEG): container finished" podID="eff69ae2-ba13-4f60-99f6-106af37b8e1c" containerID="8bf1db9505c3781de755a80768ff86c6d954dc23d3281cce7b5b0066565b482c" exitCode=0
Jul 29 18:46:14.070303 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:14.069937    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72" event={"ID":"eff69ae2-ba13-4f60-99f6-106af37b8e1c","Type":"ContainerDied","Data":"8bf1db9505c3781de755a80768ff86c6d954dc23d3281cce7b5b0066565b482c"}
Jul 29 18:46:16.083624 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:16.083589    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72" event={"ID":"eff69ae2-ba13-4f60-99f6-106af37b8e1c","Type":"ContainerStarted","Data":"eaef925567bad79e8d8e104a98dab09b0b996b00b17e3adbb7e19498cabb64ed"}
Jul 29 18:46:16.103356 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:16.103299    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72" podStartSLOduration=2.305490011 podStartE2EDuration="13.103283426s" totalImagesPullingTime="5.189100103s" totalInitContainerRuntime="5.608693312s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:46:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:46:16.102076642 +0000 UTC m=+702.053643747" watchObservedRunningTime="2026-07-29 18:46:16.103283426 +0000 UTC m=+702.054850503"
Jul 29 18:46:24.163003 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:24.162960    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:24.163532 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:24.163033    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:24.175624 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:24.175597    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:25.139610 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:25.139578    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:46:48.367819 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.367778    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"]
Jul 29 18:46:48.370423 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.370402    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.373355 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.373329    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs\""
Jul 29 18:46:48.384304 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.384273    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"]
Jul 29 18:46:48.424100 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.424059    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.424303 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.424108    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f7e739b-0e4d-4374-bed7-242691b36c41-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.424303 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.424186    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t2jz\" (UniqueName: \"kubernetes.io/projected/3f7e739b-0e4d-4374-bed7-242691b36c41-kube-api-access-9t2jz\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.424303 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.424250    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.424303 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.424271    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-kserve-provision-location\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.424303 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.424292    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.424498 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.424321    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.525236 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.525193    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t2jz\" (UniqueName: \"kubernetes.io/projected/3f7e739b-0e4d-4374-bed7-242691b36c41-kube-api-access-9t2jz\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.525422 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.525258    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.525422 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.525280    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-kserve-provision-location\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.525422 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.525306    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.525422 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.525330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.525422 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.525376    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.525422 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.525403    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f7e739b-0e4d-4374-bed7-242691b36c41-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.525797 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.525737    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.525858 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.525798    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.525858 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.525842    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.525933 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.525884    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-kserve-provision-location\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.527741 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.527715    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.528003 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.527987    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f7e739b-0e4d-4374-bed7-242691b36c41-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.533609 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.533580    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t2jz\" (UniqueName: \"kubernetes.io/projected/3f7e739b-0e4d-4374-bed7-242691b36c41-kube-api-access-9t2jz\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.681765 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.681685    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:46:48.845626 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:48.845590    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"]
Jul 29 18:46:48.849719 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:46:48.849687    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f7e739b_0e4d_4374_bed7_242691b36c41.slice/crio-1a450395020761332b29d14adce50d71a76772616f399b27c3006fbc0b5b9076 WatchSource:0}: Error finding container 1a450395020761332b29d14adce50d71a76772616f399b27c3006fbc0b5b9076: Status 404 returned error can't find the container with id 1a450395020761332b29d14adce50d71a76772616f399b27c3006fbc0b5b9076
Jul 29 18:46:49.248469 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:49.248435    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w" event={"ID":"3f7e739b-0e4d-4374-bed7-242691b36c41","Type":"ContainerStarted","Data":"cacc36867fc16e2a0d90c6697e78fecf094eb7f1ba32e41413ca16d926e85084"}
Jul 29 18:46:49.248678 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:49.248491    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w" event={"ID":"3f7e739b-0e4d-4374-bed7-242691b36c41","Type":"ContainerStarted","Data":"1a450395020761332b29d14adce50d71a76772616f399b27c3006fbc0b5b9076"}
Jul 29 18:46:50.766943 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:50.766906    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"]
Jul 29 18:46:51.265917 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:46:51.265837    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w" podUID="3f7e739b-0e4d-4374-bed7-242691b36c41" containerName="storage-initializer" containerID="cri-o://cacc36867fc16e2a0d90c6697e78fecf094eb7f1ba32e41413ca16d926e85084" gracePeriod=30
Jul 29 18:47:00.029230 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.029200    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:47:00.139749 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.139657    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t2jz\" (UniqueName: \"kubernetes.io/projected/3f7e739b-0e4d-4374-bed7-242691b36c41-kube-api-access-9t2jz\") pod \"3f7e739b-0e4d-4374-bed7-242691b36c41\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") "
Jul 29 18:47:00.139749 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.139699    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-model-cache\") pod \"3f7e739b-0e4d-4374-bed7-242691b36c41\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") "
Jul 29 18:47:00.139749 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.139744    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-dshm\") pod \"3f7e739b-0e4d-4374-bed7-242691b36c41\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") "
Jul 29 18:47:00.139998 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.139804    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-tmp-dir\") pod \"3f7e739b-0e4d-4374-bed7-242691b36c41\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") "
Jul 29 18:47:00.139998 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.139844    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-home\") pod \"3f7e739b-0e4d-4374-bed7-242691b36c41\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") "
Jul 29 18:47:00.139998 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.139883    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-kserve-provision-location\") pod \"3f7e739b-0e4d-4374-bed7-242691b36c41\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") "
Jul 29 18:47:00.139998 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.139929    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f7e739b-0e4d-4374-bed7-242691b36c41-tls-certs\") pod \"3f7e739b-0e4d-4374-bed7-242691b36c41\" (UID: \"3f7e739b-0e4d-4374-bed7-242691b36c41\") "
Jul 29 18:47:00.140199 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.140043    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-model-cache" (OuterVolumeSpecName: "model-cache") pod "3f7e739b-0e4d-4374-bed7-242691b36c41" (UID: "3f7e739b-0e4d-4374-bed7-242691b36c41"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:47:00.140199 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.140115    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3f7e739b-0e4d-4374-bed7-242691b36c41" (UID: "3f7e739b-0e4d-4374-bed7-242691b36c41"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:47:00.140199 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.140134    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-home" (OuterVolumeSpecName: "home") pod "3f7e739b-0e4d-4374-bed7-242691b36c41" (UID: "3f7e739b-0e4d-4374-bed7-242691b36c41"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:47:00.140307 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.140288    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-model-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:47:00.140341 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.140305    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-tmp-dir\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:47:00.140341 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.140318    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-home\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:47:00.142059 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.142034    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f7e739b-0e4d-4374-bed7-242691b36c41-kube-api-access-9t2jz" (OuterVolumeSpecName: "kube-api-access-9t2jz") pod "3f7e739b-0e4d-4374-bed7-242691b36c41" (UID: "3f7e739b-0e4d-4374-bed7-242691b36c41"). InnerVolumeSpecName "kube-api-access-9t2jz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:47:00.142165 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.142056    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f7e739b-0e4d-4374-bed7-242691b36c41-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3f7e739b-0e4d-4374-bed7-242691b36c41" (UID: "3f7e739b-0e4d-4374-bed7-242691b36c41"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:47:00.142254 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.142238    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-dshm" (OuterVolumeSpecName: "dshm") pod "3f7e739b-0e4d-4374-bed7-242691b36c41" (UID: "3f7e739b-0e4d-4374-bed7-242691b36c41"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:47:00.206099 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.206052    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "3f7e739b-0e4d-4374-bed7-242691b36c41" (UID: "3f7e739b-0e4d-4374-bed7-242691b36c41"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:47:00.241611 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.241578    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-kserve-provision-location\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:47:00.241611 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.241609    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f7e739b-0e4d-4374-bed7-242691b36c41-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:47:00.241611 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.241618    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t2jz\" (UniqueName: \"kubernetes.io/projected/3f7e739b-0e4d-4374-bed7-242691b36c41-kube-api-access-9t2jz\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:47:00.241850 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.241630    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f7e739b-0e4d-4374-bed7-242691b36c41-dshm\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:47:00.315066 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.315031    2579 generic.go:356] "Generic (PLEG): container finished" podID="3f7e739b-0e4d-4374-bed7-242691b36c41" containerID="cacc36867fc16e2a0d90c6697e78fecf094eb7f1ba32e41413ca16d926e85084" exitCode=0
Jul 29 18:47:00.315237 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.315082    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"
Jul 29 18:47:00.315237 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.315121    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w" event={"ID":"3f7e739b-0e4d-4374-bed7-242691b36c41","Type":"ContainerDied","Data":"cacc36867fc16e2a0d90c6697e78fecf094eb7f1ba32e41413ca16d926e85084"}
Jul 29 18:47:00.315237 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.315175    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w" event={"ID":"3f7e739b-0e4d-4374-bed7-242691b36c41","Type":"ContainerDied","Data":"1a450395020761332b29d14adce50d71a76772616f399b27c3006fbc0b5b9076"}
Jul 29 18:47:00.315237 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.315198    2579 scope.go:117] "RemoveContainer" containerID="cacc36867fc16e2a0d90c6697e78fecf094eb7f1ba32e41413ca16d926e85084"
Jul 29 18:47:00.351643 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.351606    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"]
Jul 29 18:47:00.357765 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.357724    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7f8d549bfd5gd8w"]
Jul 29 18:47:00.585803 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.585768    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f7e739b-0e4d-4374-bed7-242691b36c41" path="/var/lib/kubelet/pods/3f7e739b-0e4d-4374-bed7-242691b36c41/volumes"
Jul 29 18:47:00.735732 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.735712    2579 scope.go:117] "RemoveContainer" containerID="cacc36867fc16e2a0d90c6697e78fecf094eb7f1ba32e41413ca16d926e85084"
Jul 29 18:47:00.736084 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:47:00.736061    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cacc36867fc16e2a0d90c6697e78fecf094eb7f1ba32e41413ca16d926e85084\": container with ID starting with cacc36867fc16e2a0d90c6697e78fecf094eb7f1ba32e41413ca16d926e85084 not found: ID does not exist" containerID="cacc36867fc16e2a0d90c6697e78fecf094eb7f1ba32e41413ca16d926e85084"
Jul 29 18:47:00.736165 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:00.736094    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cacc36867fc16e2a0d90c6697e78fecf094eb7f1ba32e41413ca16d926e85084"} err="failed to get container status \"cacc36867fc16e2a0d90c6697e78fecf094eb7f1ba32e41413ca16d926e85084\": rpc error: code = NotFound desc = could not find container \"cacc36867fc16e2a0d90c6697e78fecf094eb7f1ba32e41413ca16d926e85084\": container with ID starting with cacc36867fc16e2a0d90c6697e78fecf094eb7f1ba32e41413ca16d926e85084 not found: ID does not exist"
Jul 29 18:47:22.725195 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:22.725152    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"]
Jul 29 18:47:22.725699 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:22.725441    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72" podUID="eff69ae2-ba13-4f60-99f6-106af37b8e1c" containerName="main" containerID="cri-o://eaef925567bad79e8d8e104a98dab09b0b996b00b17e3adbb7e19498cabb64ed" gracePeriod=30
Jul 29 18:47:22.986586 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:22.986514    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:47:23.041871 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.041839    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-tmp-dir\") pod \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") "
Jul 29 18:47:23.041871 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.041876    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-dshm\") pod \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") "
Jul 29 18:47:23.042097 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.041963    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-kserve-provision-location\") pod \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") "
Jul 29 18:47:23.042097 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.042016    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-model-cache\") pod \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") "
Jul 29 18:47:23.042097 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.042040    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-home\") pod \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") "
Jul 29 18:47:23.042097 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.042064    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eff69ae2-ba13-4f60-99f6-106af37b8e1c-tls-certs\") pod \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") "
Jul 29 18:47:23.042302 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.042102    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ggmt\" (UniqueName: \"kubernetes.io/projected/eff69ae2-ba13-4f60-99f6-106af37b8e1c-kube-api-access-4ggmt\") pod \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\" (UID: \"eff69ae2-ba13-4f60-99f6-106af37b8e1c\") "
Jul 29 18:47:23.042302 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.042186    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "eff69ae2-ba13-4f60-99f6-106af37b8e1c" (UID: "eff69ae2-ba13-4f60-99f6-106af37b8e1c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:47:23.042413 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.042310    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-model-cache" (OuterVolumeSpecName: "model-cache") pod "eff69ae2-ba13-4f60-99f6-106af37b8e1c" (UID: "eff69ae2-ba13-4f60-99f6-106af37b8e1c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:47:23.042413 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.042332    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-home" (OuterVolumeSpecName: "home") pod "eff69ae2-ba13-4f60-99f6-106af37b8e1c" (UID: "eff69ae2-ba13-4f60-99f6-106af37b8e1c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:47:23.042555 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.042469    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-tmp-dir\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:47:23.042555 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.042490    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-model-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:47:23.042555 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.042534    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-home\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:47:23.044222 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.044187    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-dshm" (OuterVolumeSpecName: "dshm") pod "eff69ae2-ba13-4f60-99f6-106af37b8e1c" (UID: "eff69ae2-ba13-4f60-99f6-106af37b8e1c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:47:23.044222 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.044214    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eff69ae2-ba13-4f60-99f6-106af37b8e1c-kube-api-access-4ggmt" (OuterVolumeSpecName: "kube-api-access-4ggmt") pod "eff69ae2-ba13-4f60-99f6-106af37b8e1c" (UID: "eff69ae2-ba13-4f60-99f6-106af37b8e1c"). InnerVolumeSpecName "kube-api-access-4ggmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:47:23.044600 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.044581    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eff69ae2-ba13-4f60-99f6-106af37b8e1c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eff69ae2-ba13-4f60-99f6-106af37b8e1c" (UID: "eff69ae2-ba13-4f60-99f6-106af37b8e1c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:47:23.097722 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.097670    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "eff69ae2-ba13-4f60-99f6-106af37b8e1c" (UID: "eff69ae2-ba13-4f60-99f6-106af37b8e1c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:47:23.143265 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.143229    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-dshm\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:47:23.143265 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.143262    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/eff69ae2-ba13-4f60-99f6-106af37b8e1c-kserve-provision-location\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:47:23.143418 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.143274    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eff69ae2-ba13-4f60-99f6-106af37b8e1c-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:47:23.143418 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.143284    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ggmt\" (UniqueName: \"kubernetes.io/projected/eff69ae2-ba13-4f60-99f6-106af37b8e1c-kube-api-access-4ggmt\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:47:23.426615 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.426576    2579 generic.go:356] "Generic (PLEG): container finished" podID="eff69ae2-ba13-4f60-99f6-106af37b8e1c" containerID="eaef925567bad79e8d8e104a98dab09b0b996b00b17e3adbb7e19498cabb64ed" exitCode=0
Jul 29 18:47:23.426790 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.426628    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"
Jul 29 18:47:23.426790 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.426662    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72" event={"ID":"eff69ae2-ba13-4f60-99f6-106af37b8e1c","Type":"ContainerDied","Data":"eaef925567bad79e8d8e104a98dab09b0b996b00b17e3adbb7e19498cabb64ed"}
Jul 29 18:47:23.426790 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.426717    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72" event={"ID":"eff69ae2-ba13-4f60-99f6-106af37b8e1c","Type":"ContainerDied","Data":"001cf023c31998135cad0597f48b61295d92cb7da28f9bf8c64f094c2316656c"}
Jul 29 18:47:23.426790 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.426740    2579 scope.go:117] "RemoveContainer" containerID="eaef925567bad79e8d8e104a98dab09b0b996b00b17e3adbb7e19498cabb64ed"
Jul 29 18:47:23.438700 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.438667    2579 scope.go:117] "RemoveContainer" containerID="8bf1db9505c3781de755a80768ff86c6d954dc23d3281cce7b5b0066565b482c"
Jul 29 18:47:23.450937 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.450913    2579 scope.go:117] "RemoveContainer" containerID="eaef925567bad79e8d8e104a98dab09b0b996b00b17e3adbb7e19498cabb64ed"
Jul 29 18:47:23.451251 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:47:23.451229    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaef925567bad79e8d8e104a98dab09b0b996b00b17e3adbb7e19498cabb64ed\": container with ID starting with eaef925567bad79e8d8e104a98dab09b0b996b00b17e3adbb7e19498cabb64ed not found: ID does not exist" containerID="eaef925567bad79e8d8e104a98dab09b0b996b00b17e3adbb7e19498cabb64ed"
Jul 29 18:47:23.451333 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.451264    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaef925567bad79e8d8e104a98dab09b0b996b00b17e3adbb7e19498cabb64ed"} err="failed to get container status \"eaef925567bad79e8d8e104a98dab09b0b996b00b17e3adbb7e19498cabb64ed\": rpc error: code = NotFound desc = could not find container \"eaef925567bad79e8d8e104a98dab09b0b996b00b17e3adbb7e19498cabb64ed\": container with ID starting with eaef925567bad79e8d8e104a98dab09b0b996b00b17e3adbb7e19498cabb64ed not found: ID does not exist"
Jul 29 18:47:23.451333 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.451291    2579 scope.go:117] "RemoveContainer" containerID="8bf1db9505c3781de755a80768ff86c6d954dc23d3281cce7b5b0066565b482c"
Jul 29 18:47:23.451642 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:47:23.451618    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bf1db9505c3781de755a80768ff86c6d954dc23d3281cce7b5b0066565b482c\": container with ID starting with 8bf1db9505c3781de755a80768ff86c6d954dc23d3281cce7b5b0066565b482c not found: ID does not exist" containerID="8bf1db9505c3781de755a80768ff86c6d954dc23d3281cce7b5b0066565b482c"
Jul 29 18:47:23.451707 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.451652    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bf1db9505c3781de755a80768ff86c6d954dc23d3281cce7b5b0066565b482c"} err="failed to get container status \"8bf1db9505c3781de755a80768ff86c6d954dc23d3281cce7b5b0066565b482c\": rpc error: code = NotFound desc = could not find container \"8bf1db9505c3781de755a80768ff86c6d954dc23d3281cce7b5b0066565b482c\": container with ID starting with 8bf1db9505c3781de755a80768ff86c6d954dc23d3281cce7b5b0066565b482c not found: ID does not exist"
Jul 29 18:47:23.452120 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.452100    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"]
Jul 29 18:47:23.456858 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:23.456828    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5db4fdc874-zfj72"]
Jul 29 18:47:24.586915 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:47:24.586879    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eff69ae2-ba13-4f60-99f6-106af37b8e1c" path="/var/lib/kubelet/pods/eff69ae2-ba13-4f60-99f6-106af37b8e1c/volumes"
Jul 29 18:48:25.626819 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.626779    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"]
Jul 29 18:48:25.627318 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.627171    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f7e739b-0e4d-4374-bed7-242691b36c41" containerName="storage-initializer"
Jul 29 18:48:25.627318 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.627182    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f7e739b-0e4d-4374-bed7-242691b36c41" containerName="storage-initializer"
Jul 29 18:48:25.627318 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.627203    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff69ae2-ba13-4f60-99f6-106af37b8e1c" containerName="storage-initializer"
Jul 29 18:48:25.627318 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.627208    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff69ae2-ba13-4f60-99f6-106af37b8e1c" containerName="storage-initializer"
Jul 29 18:48:25.627318 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.627221    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff69ae2-ba13-4f60-99f6-106af37b8e1c" containerName="main"
Jul 29 18:48:25.627318 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.627226    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff69ae2-ba13-4f60-99f6-106af37b8e1c" containerName="main"
Jul 29 18:48:25.627318 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.627291    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f7e739b-0e4d-4374-bed7-242691b36c41" containerName="storage-initializer"
Jul 29 18:48:25.627318 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.627303    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff69ae2-ba13-4f60-99f6-106af37b8e1c" containerName="main"
Jul 29 18:48:25.629927 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.629908    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.632472 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.632450    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-rd7g5\""
Jul 29 18:48:25.632615 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.632591    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Jul 29 18:48:25.633559 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.633533    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2cmt9\""
Jul 29 18:48:25.633675 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.633542    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 29 18:48:25.633675 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.633576    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 29 18:48:25.646080 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.646050    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"]
Jul 29 18:48:25.689804 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.689765    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-tmp\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.689804 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.689810    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ck62\" (UniqueName: \"kubernetes.io/projected/87ffee24-5381-465e-a3bf-b3006c4564d4-kube-api-access-8ck62\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.690041 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.689879    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.690041 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.689910    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-cache\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.690041 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.689963    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.690041 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.690016    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.790483 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.790443    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.790483 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.790486    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-cache\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.790755 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.790558    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.790755 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.790593    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.790755 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.790624    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-tmp\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.790755 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.790649    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ck62\" (UniqueName: \"kubernetes.io/projected/87ffee24-5381-465e-a3bf-b3006c4564d4-kube-api-access-8ck62\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.790950 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.790897    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-cache\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.791009 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.790958    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-tmp\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.791065 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.791006    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.791125 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.791089    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.793135 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.793114    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.800616 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.800588    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ck62\" (UniqueName: \"kubernetes.io/projected/87ffee24-5381-465e-a3bf-b3006c4564d4-kube-api-access-8ck62\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:25.939963 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:25.939862    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:26.088194 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:26.088161    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"]
Jul 29 18:48:26.091240 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:48:26.091204    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87ffee24_5381_465e_a3bf_b3006c4564d4.slice/crio-eeab072c219e7085d543ca65007d8925c8aab1b9876a346864b1e4161cbb9599 WatchSource:0}: Error finding container eeab072c219e7085d543ca65007d8925c8aab1b9876a346864b1e4161cbb9599: Status 404 returned error can't find the container with id eeab072c219e7085d543ca65007d8925c8aab1b9876a346864b1e4161cbb9599
Jul 29 18:48:26.729679 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:26.729644    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" event={"ID":"87ffee24-5381-465e-a3bf-b3006c4564d4","Type":"ContainerStarted","Data":"2f14fe93014a0d374a700beaa17e08604e14a5153ccbbeb64932dacb47426cf0"}
Jul 29 18:48:26.730046 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:26.729701    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" event={"ID":"87ffee24-5381-465e-a3bf-b3006c4564d4","Type":"ContainerStarted","Data":"eeab072c219e7085d543ca65007d8925c8aab1b9876a346864b1e4161cbb9599"}
Jul 29 18:48:27.736181 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:27.736142    2579 generic.go:356] "Generic (PLEG): container finished" podID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerID="2f14fe93014a0d374a700beaa17e08604e14a5153ccbbeb64932dacb47426cf0" exitCode=0
Jul 29 18:48:27.736705 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:27.736207    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" event={"ID":"87ffee24-5381-465e-a3bf-b3006c4564d4","Type":"ContainerDied","Data":"2f14fe93014a0d374a700beaa17e08604e14a5153ccbbeb64932dacb47426cf0"}
Jul 29 18:48:28.744491 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:28.744452    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" event={"ID":"87ffee24-5381-465e-a3bf-b3006c4564d4","Type":"ContainerStarted","Data":"8cfbb109c193ad714146a61cbe4aa1473591f6f03c3c6f0dd3c139b1c7e2d256"}
Jul 29 18:48:59.304428 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:59.304317    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"]
Jul 29 18:48:59.925178 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:59.925144    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" event={"ID":"87ffee24-5381-465e-a3bf-b3006c4564d4","Type":"ContainerStarted","Data":"f8058138a5234412e310b8b17c868bc726f1a822f267d526f94ab48fcd0edcdc"}
Jul 29 18:48:59.925410 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:59.925384    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" secret="" err="secret \"scheduler-ha-replicas-test-epp-sa-dockercfg-rd7g5\" not found"
Jul 29 18:48:59.925481 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:59.925442    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:48:59.928035 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:59.927979    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" podUID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 29 18:48:59.946038 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:48:59.945979    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" podStartSLOduration=2.944407992 podStartE2EDuration="34.94596095s" totalImagesPullingTime="31.219942106s" totalInitContainerRuntime="781.610852ms" isStatefulPod=false podCreationTimestamp="2026-07-29 18:48:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:48:59.945913461 +0000 UTC m=+865.897480556" watchObservedRunningTime="2026-07-29 18:48:59.94596095 +0000 UTC m=+865.897528025"
Jul 29 18:49:00.026842 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:49:00.026809    2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs: secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found
Jul 29 18:49:00.027010 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:49:00.026907    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs podName:87ffee24-5381-465e-a3bf-b3006c4564d4 nodeName:}" failed. No retries permitted until 2026-07-29 18:49:00.526886267 +0000 UTC m=+866.478453341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs") pod "scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" (UID: "87ffee24-5381-465e-a3bf-b3006c4564d4") : secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found
Jul 29 18:49:00.540389 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:49:00.540334    2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs: secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found
Jul 29 18:49:00.540830 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:49:00.540423    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs podName:87ffee24-5381-465e-a3bf-b3006c4564d4 nodeName:}" failed. No retries permitted until 2026-07-29 18:49:01.540408028 +0000 UTC m=+867.491975083 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs") pod "scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" (UID: "87ffee24-5381-465e-a3bf-b3006c4564d4") : secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found
Jul 29 18:49:00.934310 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:00.934198    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" podUID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerName="main" containerID="cri-o://8cfbb109c193ad714146a61cbe4aa1473591f6f03c3c6f0dd3c139b1c7e2d256" gracePeriod=30
Jul 29 18:49:00.934487 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:00.934246    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" podUID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerName="tokenizer" containerID="cri-o://f8058138a5234412e310b8b17c868bc726f1a822f267d526f94ab48fcd0edcdc" gracePeriod=30
Jul 29 18:49:00.935979 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:00.935939    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" podUID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 29 18:49:01.550524 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:49:01.550481    2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs: secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found
Jul 29 18:49:01.550984 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:49:01.550590    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs podName:87ffee24-5381-465e-a3bf-b3006c4564d4 nodeName:}" failed. No retries permitted until 2026-07-29 18:49:03.550567877 +0000 UTC m=+869.502134933 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs") pod "scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" (UID: "87ffee24-5381-465e-a3bf-b3006c4564d4") : secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found
Jul 29 18:49:01.942069 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:01.941982    2579 generic.go:356] "Generic (PLEG): container finished" podID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerID="8cfbb109c193ad714146a61cbe4aa1473591f6f03c3c6f0dd3c139b1c7e2d256" exitCode=0
Jul 29 18:49:01.942069 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:01.942051    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" event={"ID":"87ffee24-5381-465e-a3bf-b3006c4564d4","Type":"ContainerDied","Data":"8cfbb109c193ad714146a61cbe4aa1473591f6f03c3c6f0dd3c139b1c7e2d256"}
Jul 29 18:49:03.570089 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:49:03.570053    2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs: secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found
Jul 29 18:49:03.570494 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:49:03.570148    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs podName:87ffee24-5381-465e-a3bf-b3006c4564d4 nodeName:}" failed. No retries permitted until 2026-07-29 18:49:07.570129268 +0000 UTC m=+873.521696323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs") pod "scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" (UID: "87ffee24-5381-465e-a3bf-b3006c4564d4") : secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found
Jul 29 18:49:05.940177 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:05.940138    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:49:07.609684 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:49:07.609640    2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs: secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found
Jul 29 18:49:07.610097 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:49:07.609725    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs podName:87ffee24-5381-465e-a3bf-b3006c4564d4 nodeName:}" failed. No retries permitted until 2026-07-29 18:49:15.609706701 +0000 UTC m=+881.561273776 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs") pod "scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" (UID: "87ffee24-5381-465e-a3bf-b3006c4564d4") : secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found
Jul 29 18:49:10.935860 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:49:10.935829    2579 logging.go:55] [core] [Channel #23 SubChannel #24]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.40:9003", ServerName: "10.132.0.40:9003", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.40:9003: connect: connection refused"
Jul 29 18:49:11.935395 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:11.935344    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" podUID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.132.0.40:9003\" within 1s: context deadline exceeded"
Jul 29 18:49:15.690356 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:49:15.690319    2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs: secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found
Jul 29 18:49:15.690758 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:49:15.690408    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs podName:87ffee24-5381-465e-a3bf-b3006c4564d4 nodeName:}" failed. No retries permitted until 2026-07-29 18:49:31.690390697 +0000 UTC m=+897.641957752 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs") pod "scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" (UID: "87ffee24-5381-465e-a3bf-b3006c4564d4") : secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found
Jul 29 18:49:20.935130 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:49:20.935092    2579 logging.go:55] [core] [Channel #25 SubChannel #26]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.40:9003", ServerName: "10.132.0.40:9003", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.40:9003: connect: connection refused"
Jul 29 18:49:21.935547 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:21.935474    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" podUID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.132.0.40:9003\" within 1s: context deadline exceeded"
Jul 29 18:49:21.936029 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:49:21.935713    2579 logging.go:55] [core] [Channel #25 SubChannel #26]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.40:9003", ServerName: "10.132.0.40:9003", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.40:9003: operation was canceled"
Jul 29 18:49:30.935469 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:49:30.935386    2579 logging.go:55] [core] [Channel #27 SubChannel #28]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.40:9003", ServerName: "10.132.0.40:9003", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.40:9003: connect: connection refused"
Jul 29 18:49:31.091672 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.091638    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf_87ffee24-5381-465e-a3bf-b3006c4564d4/tokenizer/0.log"
Jul 29 18:49:31.092342 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.092320    2579 generic.go:356] "Generic (PLEG): container finished" podID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerID="f8058138a5234412e310b8b17c868bc726f1a822f267d526f94ab48fcd0edcdc" exitCode=137
Jul 29 18:49:31.092439 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.092403    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" event={"ID":"87ffee24-5381-465e-a3bf-b3006c4564d4","Type":"ContainerDied","Data":"f8058138a5234412e310b8b17c868bc726f1a822f267d526f94ab48fcd0edcdc"}
Jul 29 18:49:31.594717 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.594689    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf_87ffee24-5381-465e-a3bf-b3006c4564d4/tokenizer/0.log"
Jul 29 18:49:31.597047 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.597025    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:49:31.744080 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.743974    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-cache\") pod \"87ffee24-5381-465e-a3bf-b3006c4564d4\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") "
Jul 29 18:49:31.744080 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.744037    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-tmp\") pod \"87ffee24-5381-465e-a3bf-b3006c4564d4\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") "
Jul 29 18:49:31.744080 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.744071    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ck62\" (UniqueName: \"kubernetes.io/projected/87ffee24-5381-465e-a3bf-b3006c4564d4-kube-api-access-8ck62\") pod \"87ffee24-5381-465e-a3bf-b3006c4564d4\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") "
Jul 29 18:49:31.744373 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.744098    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-kserve-provision-location\") pod \"87ffee24-5381-465e-a3bf-b3006c4564d4\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") "
Jul 29 18:49:31.744373 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.744135    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs\") pod \"87ffee24-5381-465e-a3bf-b3006c4564d4\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") "
Jul 29 18:49:31.744373 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.744241    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-uds\") pod \"87ffee24-5381-465e-a3bf-b3006c4564d4\" (UID: \"87ffee24-5381-465e-a3bf-b3006c4564d4\") "
Jul 29 18:49:31.744373 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.744334    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "87ffee24-5381-465e-a3bf-b3006c4564d4" (UID: "87ffee24-5381-465e-a3bf-b3006c4564d4"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:49:31.744595 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.744473    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "87ffee24-5381-465e-a3bf-b3006c4564d4" (UID: "87ffee24-5381-465e-a3bf-b3006c4564d4"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:49:31.744739 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.744719    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:49:31.744806 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.744731    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "87ffee24-5381-465e-a3bf-b3006c4564d4" (UID: "87ffee24-5381-465e-a3bf-b3006c4564d4"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:49:31.744806 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.744745    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-tmp\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:49:31.745050 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.745020    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "87ffee24-5381-465e-a3bf-b3006c4564d4" (UID: "87ffee24-5381-465e-a3bf-b3006c4564d4"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:49:31.746387 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.746368    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87ffee24-5381-465e-a3bf-b3006c4564d4-kube-api-access-8ck62" (OuterVolumeSpecName: "kube-api-access-8ck62") pod "87ffee24-5381-465e-a3bf-b3006c4564d4" (UID: "87ffee24-5381-465e-a3bf-b3006c4564d4"). InnerVolumeSpecName "kube-api-access-8ck62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:49:31.746387 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.746381    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "87ffee24-5381-465e-a3bf-b3006c4564d4" (UID: "87ffee24-5381-465e-a3bf-b3006c4564d4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:49:31.845997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.845955    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-tokenizer-uds\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:49:31.845997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.845989    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ck62\" (UniqueName: \"kubernetes.io/projected/87ffee24-5381-465e-a3bf-b3006c4564d4-kube-api-access-8ck62\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:49:31.845997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.846000    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87ffee24-5381-465e-a3bf-b3006c4564d4-kserve-provision-location\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:49:31.845997 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.846009    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87ffee24-5381-465e-a3bf-b3006c4564d4-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:49:31.935145 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:31.935075    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" podUID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.132.0.40:9003\" within 1s: context deadline exceeded"
Jul 29 18:49:32.099215 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:32.099182    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf_87ffee24-5381-465e-a3bf-b3006c4564d4/tokenizer/0.log"
Jul 29 18:49:32.100040 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:32.100024    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"
Jul 29 18:49:32.100125 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:32.100040    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf" event={"ID":"87ffee24-5381-465e-a3bf-b3006c4564d4","Type":"ContainerDied","Data":"eeab072c219e7085d543ca65007d8925c8aab1b9876a346864b1e4161cbb9599"}
Jul 29 18:49:32.100167 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:32.100142    2579 scope.go:117] "RemoveContainer" containerID="f8058138a5234412e310b8b17c868bc726f1a822f267d526f94ab48fcd0edcdc"
Jul 29 18:49:32.109710 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:32.109692    2579 scope.go:117] "RemoveContainer" containerID="8cfbb109c193ad714146a61cbe4aa1473591f6f03c3c6f0dd3c139b1c7e2d256"
Jul 29 18:49:32.118333 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:32.118310    2579 scope.go:117] "RemoveContainer" containerID="2f14fe93014a0d374a700beaa17e08604e14a5153ccbbeb64932dacb47426cf0"
Jul 29 18:49:32.124553 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:32.124477    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"]
Jul 29 18:49:32.126958 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:32.126929    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-856ddff8f6mf"]
Jul 29 18:49:32.585071 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:32.585038    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87ffee24-5381-465e-a3bf-b3006c4564d4" path="/var/lib/kubelet/pods/87ffee24-5381-465e-a3bf-b3006c4564d4/volumes"
Jul 29 18:49:34.474673 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:34.474643    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 18:49:34.476322 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:34.476302    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 18:49:50.910311 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:50.910266    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"]
Jul 29 18:49:50.910990 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:50.910724    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerName="storage-initializer"
Jul 29 18:49:50.910990 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:50.910740    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerName="storage-initializer"
Jul 29 18:49:50.910990 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:50.910767    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerName="tokenizer"
Jul 29 18:49:50.910990 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:50.910776    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerName="tokenizer"
Jul 29 18:49:50.910990 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:50.910789    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerName="main"
Jul 29 18:49:50.910990 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:50.910796    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerName="main"
Jul 29 18:49:50.910990 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:50.910881    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerName="main"
Jul 29 18:49:50.910990 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:50.910894    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="87ffee24-5381-465e-a3bf-b3006c4564d4" containerName="tokenizer"
Jul 29 18:49:50.915115 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:50.915095    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:50.917933 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:50.917906    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-kserve-self-signed-certs\""
Jul 29 18:49:50.918078 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:50.917962    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-epp-sa-dockercfg-sgqj9\""
Jul 29 18:49:50.918078 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:50.918022    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2cmt9\""
Jul 29 18:49:50.918223 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:50.918189    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 29 18:49:50.918610 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:50.918592    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 29 18:49:50.925904 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:50.925882    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"]
Jul 29 18:49:51.011881 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.011848    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.011881 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.011885    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.012155 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.011917    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dttn4\" (UniqueName: \"kubernetes.io/projected/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-kube-api-access-dttn4\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.012155 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.012049    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.012155 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.012139    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.012288 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.012176    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.113136 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.113105    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.113334 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.113162    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.113334 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.113186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.113334 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.113210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.113334 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.113226    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.113334 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.113247    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dttn4\" (UniqueName: \"kubernetes.io/projected/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-kube-api-access-dttn4\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.113617 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.113579    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.113617 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.113609    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.113717 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.113653    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.113717 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.113693    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.115913 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.115886    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.126899 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.126868    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dttn4\" (UniqueName: \"kubernetes.io/projected/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-kube-api-access-dttn4\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.226795 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.226693    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:51.372785 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:51.372752    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"]
Jul 29 18:49:51.374460 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:49:51.374432    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fe310d2_96e8_42dc_ac78_ae9687ee5a72.slice/crio-3c7c162a9f45f34d6987f4d531f420de3f26f1e3bf22892326e22308d36c8afe WatchSource:0}: Error finding container 3c7c162a9f45f34d6987f4d531f420de3f26f1e3bf22892326e22308d36c8afe: Status 404 returned error can't find the container with id 3c7c162a9f45f34d6987f4d531f420de3f26f1e3bf22892326e22308d36c8afe
Jul 29 18:49:52.202958 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:52.202921    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc" event={"ID":"3fe310d2-96e8-42dc-ac78-ae9687ee5a72","Type":"ContainerStarted","Data":"8b721d1e6c927cfe6e7f8d9a0d1ba85d1c4263c5c4c620d19b5a7478591f3656"}
Jul 29 18:49:52.203427 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:52.202980    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc" event={"ID":"3fe310d2-96e8-42dc-ac78-ae9687ee5a72","Type":"ContainerStarted","Data":"3c7c162a9f45f34d6987f4d531f420de3f26f1e3bf22892326e22308d36c8afe"}
Jul 29 18:49:53.209072 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:53.209040    2579 generic.go:356] "Generic (PLEG): container finished" podID="3fe310d2-96e8-42dc-ac78-ae9687ee5a72" containerID="8b721d1e6c927cfe6e7f8d9a0d1ba85d1c4263c5c4c620d19b5a7478591f3656" exitCode=0
Jul 29 18:49:53.209474 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:53.209132    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc" event={"ID":"3fe310d2-96e8-42dc-ac78-ae9687ee5a72","Type":"ContainerDied","Data":"8b721d1e6c927cfe6e7f8d9a0d1ba85d1c4263c5c4c620d19b5a7478591f3656"}
Jul 29 18:49:54.217199 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:54.217164    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc" event={"ID":"3fe310d2-96e8-42dc-ac78-ae9687ee5a72","Type":"ContainerStarted","Data":"d17051053250d4efa94142b32c3181c26da74cb10d8b1a0e100dcf10ab7c11ee"}
Jul 29 18:49:54.217627 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:54.217220    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc" event={"ID":"3fe310d2-96e8-42dc-ac78-ae9687ee5a72","Type":"ContainerStarted","Data":"e1a53e1513523fdaef107792f104c9e568b020d95a0257341a6921adfb76f444"}
Jul 29 18:49:54.217627 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:54.217269    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:49:54.238136 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:49:54.238061    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc" podStartSLOduration=3.357055702 podStartE2EDuration="4.238041632s" totalImagesPullingTime="0s" totalInitContainerRuntime="880.98593ms" isStatefulPod=false podCreationTimestamp="2026-07-29 18:49:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:49:54.236285883 +0000 UTC m=+920.187852988" watchObservedRunningTime="2026-07-29 18:49:54.238041632 +0000 UTC m=+920.189608708"
Jul 29 18:50:01.227651 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:50:01.227608    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:50:01.228207 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:50:01.227707    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:50:01.230635 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:50:01.230605    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:50:01.258218 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:50:01.258187    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:50:32.264224 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:50:32.264187    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:52:12.021648 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:12.021609    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"]
Jul 29 18:52:12.022165 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:12.021872    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc" podUID="3fe310d2-96e8-42dc-ac78-ae9687ee5a72" containerName="main" containerID="cri-o://e1a53e1513523fdaef107792f104c9e568b020d95a0257341a6921adfb76f444" gracePeriod=30
Jul 29 18:52:12.022165 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:12.021915    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc" podUID="3fe310d2-96e8-42dc-ac78-ae9687ee5a72" containerName="tokenizer" containerID="cri-o://d17051053250d4efa94142b32c3181c26da74cb10d8b1a0e100dcf10ab7c11ee" gracePeriod=30
Jul 29 18:52:12.263606 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:52:12.263573    2579 logging.go:55] [core] [Channel #101 SubChannel #102]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.41:9003", ServerName: "10.132.0.41:9003", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.41:9003: connect: connection refused"
Jul 29 18:52:12.912765 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:12.912727    2579 generic.go:356] "Generic (PLEG): container finished" podID="3fe310d2-96e8-42dc-ac78-ae9687ee5a72" containerID="e1a53e1513523fdaef107792f104c9e568b020d95a0257341a6921adfb76f444" exitCode=0
Jul 29 18:52:12.912942 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:12.912803    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc" event={"ID":"3fe310d2-96e8-42dc-ac78-ae9687ee5a72","Type":"ContainerDied","Data":"e1a53e1513523fdaef107792f104c9e568b020d95a0257341a6921adfb76f444"}
Jul 29 18:52:13.263475 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.263432    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc" podUID="3fe310d2-96e8-42dc-ac78-ae9687ee5a72" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.132.0.41:9003\" within 1s: context deadline exceeded"
Jul 29 18:52:13.488064 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.488028    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:52:13.641213 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.641170    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-tmp\") pod \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") "
Jul 29 18:52:13.641213 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.641207    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-uds\") pod \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") "
Jul 29 18:52:13.641485 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.641229    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tls-certs\") pod \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") "
Jul 29 18:52:13.641485 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.641277    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dttn4\" (UniqueName: \"kubernetes.io/projected/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-kube-api-access-dttn4\") pod \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") "
Jul 29 18:52:13.641485 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.641308    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-kserve-provision-location\") pod \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") "
Jul 29 18:52:13.641485 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.641335    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-cache\") pod \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\" (UID: \"3fe310d2-96e8-42dc-ac78-ae9687ee5a72\") "
Jul 29 18:52:13.641742 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.641551    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "3fe310d2-96e8-42dc-ac78-ae9687ee5a72" (UID: "3fe310d2-96e8-42dc-ac78-ae9687ee5a72"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:52:13.641742 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.641707    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "3fe310d2-96e8-42dc-ac78-ae9687ee5a72" (UID: "3fe310d2-96e8-42dc-ac78-ae9687ee5a72"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:52:13.641742 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.641730    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "3fe310d2-96e8-42dc-ac78-ae9687ee5a72" (UID: "3fe310d2-96e8-42dc-ac78-ae9687ee5a72"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:52:13.641905 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.641797    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-uds\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:52:13.642261 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.642239    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "3fe310d2-96e8-42dc-ac78-ae9687ee5a72" (UID: "3fe310d2-96e8-42dc-ac78-ae9687ee5a72"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:52:13.643475 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.643450    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3fe310d2-96e8-42dc-ac78-ae9687ee5a72" (UID: "3fe310d2-96e8-42dc-ac78-ae9687ee5a72"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:52:13.643709 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.643685    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-kube-api-access-dttn4" (OuterVolumeSpecName: "kube-api-access-dttn4") pod "3fe310d2-96e8-42dc-ac78-ae9687ee5a72" (UID: "3fe310d2-96e8-42dc-ac78-ae9687ee5a72"). InnerVolumeSpecName "kube-api-access-dttn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:52:13.742285 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.742239    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:52:13.742285 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.742278    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tokenizer-tmp\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:52:13.742285 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.742289    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:52:13.742285 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.742297    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dttn4\" (UniqueName: \"kubernetes.io/projected/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-kube-api-access-dttn4\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:52:13.742605 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.742307    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3fe310d2-96e8-42dc-ac78-ae9687ee5a72-kserve-provision-location\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:52:13.920183 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.920088    2579 generic.go:356] "Generic (PLEG): container finished" podID="3fe310d2-96e8-42dc-ac78-ae9687ee5a72" containerID="d17051053250d4efa94142b32c3181c26da74cb10d8b1a0e100dcf10ab7c11ee" exitCode=0
Jul 29 18:52:13.920183 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.920126    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc" event={"ID":"3fe310d2-96e8-42dc-ac78-ae9687ee5a72","Type":"ContainerDied","Data":"d17051053250d4efa94142b32c3181c26da74cb10d8b1a0e100dcf10ab7c11ee"}
Jul 29 18:52:13.920183 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.920157    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"
Jul 29 18:52:13.920430 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.920191    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc" event={"ID":"3fe310d2-96e8-42dc-ac78-ae9687ee5a72","Type":"ContainerDied","Data":"3c7c162a9f45f34d6987f4d531f420de3f26f1e3bf22892326e22308d36c8afe"}
Jul 29 18:52:13.920430 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.920214    2579 scope.go:117] "RemoveContainer" containerID="d17051053250d4efa94142b32c3181c26da74cb10d8b1a0e100dcf10ab7c11ee"
Jul 29 18:52:13.929330 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.929312    2579 scope.go:117] "RemoveContainer" containerID="e1a53e1513523fdaef107792f104c9e568b020d95a0257341a6921adfb76f444"
Jul 29 18:52:13.937454 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.937434    2579 scope.go:117] "RemoveContainer" containerID="8b721d1e6c927cfe6e7f8d9a0d1ba85d1c4263c5c4c620d19b5a7478591f3656"
Jul 29 18:52:13.943671 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.943641    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"]
Jul 29 18:52:13.946851 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.946820    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-z8ntc"]
Jul 29 18:52:13.947289 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.947271    2579 scope.go:117] "RemoveContainer" containerID="d17051053250d4efa94142b32c3181c26da74cb10d8b1a0e100dcf10ab7c11ee"
Jul 29 18:52:13.947648 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:52:13.947628    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d17051053250d4efa94142b32c3181c26da74cb10d8b1a0e100dcf10ab7c11ee\": container with ID starting with d17051053250d4efa94142b32c3181c26da74cb10d8b1a0e100dcf10ab7c11ee not found: ID does not exist" containerID="d17051053250d4efa94142b32c3181c26da74cb10d8b1a0e100dcf10ab7c11ee"
Jul 29 18:52:13.947717 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.947660    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d17051053250d4efa94142b32c3181c26da74cb10d8b1a0e100dcf10ab7c11ee"} err="failed to get container status \"d17051053250d4efa94142b32c3181c26da74cb10d8b1a0e100dcf10ab7c11ee\": rpc error: code = NotFound desc = could not find container \"d17051053250d4efa94142b32c3181c26da74cb10d8b1a0e100dcf10ab7c11ee\": container with ID starting with d17051053250d4efa94142b32c3181c26da74cb10d8b1a0e100dcf10ab7c11ee not found: ID does not exist"
Jul 29 18:52:13.947717 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.947679    2579 scope.go:117] "RemoveContainer" containerID="e1a53e1513523fdaef107792f104c9e568b020d95a0257341a6921adfb76f444"
Jul 29 18:52:13.947950 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:52:13.947931    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1a53e1513523fdaef107792f104c9e568b020d95a0257341a6921adfb76f444\": container with ID starting with e1a53e1513523fdaef107792f104c9e568b020d95a0257341a6921adfb76f444 not found: ID does not exist" containerID="e1a53e1513523fdaef107792f104c9e568b020d95a0257341a6921adfb76f444"
Jul 29 18:52:13.948017 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.947963    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1a53e1513523fdaef107792f104c9e568b020d95a0257341a6921adfb76f444"} err="failed to get container status \"e1a53e1513523fdaef107792f104c9e568b020d95a0257341a6921adfb76f444\": rpc error: code = NotFound desc = could not find container \"e1a53e1513523fdaef107792f104c9e568b020d95a0257341a6921adfb76f444\": container with ID starting with e1a53e1513523fdaef107792f104c9e568b020d95a0257341a6921adfb76f444 not found: ID does not exist"
Jul 29 18:52:13.948017 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.947986    2579 scope.go:117] "RemoveContainer" containerID="8b721d1e6c927cfe6e7f8d9a0d1ba85d1c4263c5c4c620d19b5a7478591f3656"
Jul 29 18:52:13.948259 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:52:13.948240    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b721d1e6c927cfe6e7f8d9a0d1ba85d1c4263c5c4c620d19b5a7478591f3656\": container with ID starting with 8b721d1e6c927cfe6e7f8d9a0d1ba85d1c4263c5c4c620d19b5a7478591f3656 not found: ID does not exist" containerID="8b721d1e6c927cfe6e7f8d9a0d1ba85d1c4263c5c4c620d19b5a7478591f3656"
Jul 29 18:52:13.948319 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:13.948264    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b721d1e6c927cfe6e7f8d9a0d1ba85d1c4263c5c4c620d19b5a7478591f3656"} err="failed to get container status \"8b721d1e6c927cfe6e7f8d9a0d1ba85d1c4263c5c4c620d19b5a7478591f3656\": rpc error: code = NotFound desc = could not find container \"8b721d1e6c927cfe6e7f8d9a0d1ba85d1c4263c5c4c620d19b5a7478591f3656\": container with ID starting with 8b721d1e6c927cfe6e7f8d9a0d1ba85d1c4263c5c4c620d19b5a7478591f3656 not found: ID does not exist"
Jul 29 18:52:14.586089 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:14.586045    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fe310d2-96e8-42dc-ac78-ae9687ee5a72" path="/var/lib/kubelet/pods/3fe310d2-96e8-42dc-ac78-ae9687ee5a72/volumes"
Jul 29 18:52:29.774898 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.774817    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"]
Jul 29 18:52:29.775429 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.775410    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fe310d2-96e8-42dc-ac78-ae9687ee5a72" containerName="tokenizer"
Jul 29 18:52:29.775478 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.775434    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fe310d2-96e8-42dc-ac78-ae9687ee5a72" containerName="tokenizer"
Jul 29 18:52:29.775478 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.775450    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fe310d2-96e8-42dc-ac78-ae9687ee5a72" containerName="storage-initializer"
Jul 29 18:52:29.775478 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.775460    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fe310d2-96e8-42dc-ac78-ae9687ee5a72" containerName="storage-initializer"
Jul 29 18:52:29.775597 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.775485    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fe310d2-96e8-42dc-ac78-ae9687ee5a72" containerName="main"
Jul 29 18:52:29.775597 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.775494    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fe310d2-96e8-42dc-ac78-ae9687ee5a72" containerName="main"
Jul 29 18:52:29.775657 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.775631    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fe310d2-96e8-42dc-ac78-ae9687ee5a72" containerName="tokenizer"
Jul 29 18:52:29.775657 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.775653    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fe310d2-96e8-42dc-ac78-ae9687ee5a72" containerName="main"
Jul 29 18:52:29.778705 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.778680    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.782499 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.782473    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 29 18:52:29.782499 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.782489    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-epp-sa-dockercfg-r6bcv\""
Jul 29 18:52:29.782727 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.782474    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-kserve-self-signed-certs\""
Jul 29 18:52:29.782727 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.782523    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 29 18:52:29.782727 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.782478    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2cmt9\""
Jul 29 18:52:29.788021 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.787992    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"]
Jul 29 18:52:29.891420 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.891376    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c7a1339-1b48-41f4-b75d-97f404d43596-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.891644 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.891428    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.891644 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.891485    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.891644 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.891608    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.891644 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.891641    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.891818 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.891676    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6fh7\" (UniqueName: \"kubernetes.io/projected/8c7a1339-1b48-41f4-b75d-97f404d43596-kube-api-access-g6fh7\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.992554 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.992522    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.992758 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.992572    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.992758 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.992629    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.992758 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.992649    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.992758 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.992683    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6fh7\" (UniqueName: \"kubernetes.io/projected/8c7a1339-1b48-41f4-b75d-97f404d43596-kube-api-access-g6fh7\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.992758 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.992726    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c7a1339-1b48-41f4-b75d-97f404d43596-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.993016 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.992990    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.993075 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.993030    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.993272 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.993246    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.993407 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.993370    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:29.995348 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:29.995329    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c7a1339-1b48-41f4-b75d-97f404d43596-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:30.001902 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:30.001872    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6fh7\" (UniqueName: \"kubernetes.io/projected/8c7a1339-1b48-41f4-b75d-97f404d43596-kube-api-access-g6fh7\") pod \"stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:30.090158 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:30.090033    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:30.237331 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:30.237300    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"]
Jul 29 18:52:30.239288 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:52:30.239254    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c7a1339_1b48_41f4_b75d_97f404d43596.slice/crio-db3b501c20963403799cb0c8762c9c478af317e870f157a77d6d3d4d5714043a WatchSource:0}: Error finding container db3b501c20963403799cb0c8762c9c478af317e870f157a77d6d3d4d5714043a: Status 404 returned error can't find the container with id db3b501c20963403799cb0c8762c9c478af317e870f157a77d6d3d4d5714043a
Jul 29 18:52:31.003601 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:31.003568    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz" event={"ID":"8c7a1339-1b48-41f4-b75d-97f404d43596","Type":"ContainerStarted","Data":"a0720cca34b0838a80bbfa16537ce48a69eb0d23dfcf8043d9faed902836aae9"}
Jul 29 18:52:31.004053 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:31.003663    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz" event={"ID":"8c7a1339-1b48-41f4-b75d-97f404d43596","Type":"ContainerStarted","Data":"db3b501c20963403799cb0c8762c9c478af317e870f157a77d6d3d4d5714043a"}
Jul 29 18:52:32.010433 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:32.010399    2579 generic.go:356] "Generic (PLEG): container finished" podID="8c7a1339-1b48-41f4-b75d-97f404d43596" containerID="a0720cca34b0838a80bbfa16537ce48a69eb0d23dfcf8043d9faed902836aae9" exitCode=0
Jul 29 18:52:32.010861 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:32.010489    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz" event={"ID":"8c7a1339-1b48-41f4-b75d-97f404d43596","Type":"ContainerDied","Data":"a0720cca34b0838a80bbfa16537ce48a69eb0d23dfcf8043d9faed902836aae9"}
Jul 29 18:52:33.018496 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:33.018459    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz" event={"ID":"8c7a1339-1b48-41f4-b75d-97f404d43596","Type":"ContainerStarted","Data":"4c591ead22cae6e5b4488ff86e5053c699d74867767cc5c4c4b7750e175beea5"}
Jul 29 18:52:33.018946 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:33.018536    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz" event={"ID":"8c7a1339-1b48-41f4-b75d-97f404d43596","Type":"ContainerStarted","Data":"faf1710999ae1a05f25465ade721308a144578a5a0f36f3f5e73d758ef2c9645"}
Jul 29 18:52:33.018946 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:33.018626    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:33.040562 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:33.040490    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz" podStartSLOduration=3.186679252 podStartE2EDuration="4.040471638s" totalImagesPullingTime="0s" totalInitContainerRuntime="853.792386ms" isStatefulPod=false podCreationTimestamp="2026-07-29 18:52:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:52:33.0402923 +0000 UTC m=+1078.991859376" watchObservedRunningTime="2026-07-29 18:52:33.040471638 +0000 UTC m=+1078.992038715"
Jul 29 18:52:40.091216 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:40.091158    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:40.091656 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:40.091252    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:40.094137 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:40.094109    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:52:41.065847 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:52:41.065813    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:53:02.070883 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:02.070848    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:53:37.812121 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:37.812085    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"]
Jul 29 18:53:37.815876 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:37.815852    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:37.818725 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:37.818701    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-test-epp-sa-dockercfg-rhw5n\""
Jul 29 18:53:37.818875 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:37.818821    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-test-kserve-self-signed-certs\""
Jul 29 18:53:37.827044 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:37.826997    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"]
Jul 29 18:53:37.910525 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:37.910467    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/708ef420-37aa-4e0b-bc7a-9fe92379b472-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:37.910713 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:37.910548    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szppj\" (UniqueName: \"kubernetes.io/projected/708ef420-37aa-4e0b-bc7a-9fe92379b472-kube-api-access-szppj\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:37.910713 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:37.910603    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-tmp\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:37.910713 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:37.910668    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-cache\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:37.910872 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:37.910733    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:37.910872 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:37.910782    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:38.011487 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:38.011434    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/708ef420-37aa-4e0b-bc7a-9fe92379b472-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:38.011731 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:38.011539    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szppj\" (UniqueName: \"kubernetes.io/projected/708ef420-37aa-4e0b-bc7a-9fe92379b472-kube-api-access-szppj\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:38.011731 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:38.011571    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-tmp\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:38.011731 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:38.011618    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-cache\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:38.011731 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:38.011682    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:38.011731 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:38.011723    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:38.012096 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:38.012071    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-tmp\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:38.012158 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:38.012097    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:38.012195 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:38.012149    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-cache\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:38.012195 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:38.012168    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:38.014047 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:38.014026    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/708ef420-37aa-4e0b-bc7a-9fe92379b472-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:38.020281 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:38.020252    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szppj\" (UniqueName: \"kubernetes.io/projected/708ef420-37aa-4e0b-bc7a-9fe92379b472-kube-api-access-szppj\") pod \"custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:38.125952 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:38.125839    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:38.277314 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:38.277288    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"]
Jul 29 18:53:38.279027 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:53:38.278998    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod708ef420_37aa_4e0b_bc7a_9fe92379b472.slice/crio-62503a4990f00ed38d5df249d6f3c580ac926aaeded12497b0623430a3679e13 WatchSource:0}: Error finding container 62503a4990f00ed38d5df249d6f3c580ac926aaeded12497b0623430a3679e13: Status 404 returned error can't find the container with id 62503a4990f00ed38d5df249d6f3c580ac926aaeded12497b0623430a3679e13
Jul 29 18:53:38.362179 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:38.362143    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw" event={"ID":"708ef420-37aa-4e0b-bc7a-9fe92379b472","Type":"ContainerStarted","Data":"c1b5606b1733c6a3939d299c46136f8a04a1ef87c8415b360a4db28ce2a06127"}
Jul 29 18:53:38.362342 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:38.362210    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw" event={"ID":"708ef420-37aa-4e0b-bc7a-9fe92379b472","Type":"ContainerStarted","Data":"62503a4990f00ed38d5df249d6f3c580ac926aaeded12497b0623430a3679e13"}
Jul 29 18:53:39.368041 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:39.368003    2579 generic.go:356] "Generic (PLEG): container finished" podID="708ef420-37aa-4e0b-bc7a-9fe92379b472" containerID="c1b5606b1733c6a3939d299c46136f8a04a1ef87c8415b360a4db28ce2a06127" exitCode=0
Jul 29 18:53:39.368562 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:39.368076    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw" event={"ID":"708ef420-37aa-4e0b-bc7a-9fe92379b472","Type":"ContainerDied","Data":"c1b5606b1733c6a3939d299c46136f8a04a1ef87c8415b360a4db28ce2a06127"}
Jul 29 18:53:40.375922 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:40.375886    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw" event={"ID":"708ef420-37aa-4e0b-bc7a-9fe92379b472","Type":"ContainerStarted","Data":"b7c0487682b7798f4dd02c67f8ae8e2d463b118a5b12f8fdcae1c851407136c5"}
Jul 29 18:53:40.376318 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:40.375946    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw" event={"ID":"708ef420-37aa-4e0b-bc7a-9fe92379b472","Type":"ContainerStarted","Data":"c21c0c951b43074516fb53312031a0ce058bce1877553280d562d2fe52dfefa2"}
Jul 29 18:53:40.376318 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:40.376019    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:40.398876 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:40.398811    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw" podStartSLOduration=2.614721422 podStartE2EDuration="3.398791393s" totalImagesPullingTime="0s" totalInitContainerRuntime="784.069971ms" isStatefulPod=false podCreationTimestamp="2026-07-29 18:53:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:53:40.396039103 +0000 UTC m=+1146.347606180" watchObservedRunningTime="2026-07-29 18:53:40.398791393 +0000 UTC m=+1146.350358469"
Jul 29 18:53:48.126735 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:48.126688    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:48.127227 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:48.126792    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:48.129613 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:48.129582    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:53:48.431592 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:53:48.431492    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:54:09.438162 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:09.438126    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:54:20.821357 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:20.821310    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"]
Jul 29 18:54:20.821874 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:20.821632    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz" podUID="8c7a1339-1b48-41f4-b75d-97f404d43596" containerName="main" containerID="cri-o://faf1710999ae1a05f25465ade721308a144578a5a0f36f3f5e73d758ef2c9645" gracePeriod=30
Jul 29 18:54:20.821874 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:20.821715    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz" podUID="8c7a1339-1b48-41f4-b75d-97f404d43596" containerName="tokenizer" containerID="cri-o://4c591ead22cae6e5b4488ff86e5053c699d74867767cc5c4c4b7750e175beea5" gracePeriod=30
Jul 29 18:54:21.065192 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:21.065142    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz" podUID="8c7a1339-1b48-41f4-b75d-97f404d43596" containerName="tokenizer" probeResult="failure" output="Get \"http://10.132.0.42:8082/healthz\": dial tcp 10.132.0.42:8082: connect: connection refused"
Jul 29 18:54:21.605408 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:21.605374    2579 generic.go:356] "Generic (PLEG): container finished" podID="8c7a1339-1b48-41f4-b75d-97f404d43596" containerID="faf1710999ae1a05f25465ade721308a144578a5a0f36f3f5e73d758ef2c9645" exitCode=0
Jul 29 18:54:21.605616 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:21.605437    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz" event={"ID":"8c7a1339-1b48-41f4-b75d-97f404d43596","Type":"ContainerDied","Data":"faf1710999ae1a05f25465ade721308a144578a5a0f36f3f5e73d758ef2c9645"}
Jul 29 18:54:22.070524 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:54:22.070476    2579 logging.go:55] [core] [Channel #178 SubChannel #179]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.42:9003", ServerName: "10.132.0.42:9003", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.42:9003: connect: connection refused"
Jul 29 18:54:22.285949 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.285918    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:54:22.335316 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.335226    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c7a1339-1b48-41f4-b75d-97f404d43596-tls-certs\") pod \"8c7a1339-1b48-41f4-b75d-97f404d43596\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") "
Jul 29 18:54:22.335316 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.335279    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-tmp\") pod \"8c7a1339-1b48-41f4-b75d-97f404d43596\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") "
Jul 29 18:54:22.335575 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.335361    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-cache\") pod \"8c7a1339-1b48-41f4-b75d-97f404d43596\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") "
Jul 29 18:54:22.335575 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.335394    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6fh7\" (UniqueName: \"kubernetes.io/projected/8c7a1339-1b48-41f4-b75d-97f404d43596-kube-api-access-g6fh7\") pod \"8c7a1339-1b48-41f4-b75d-97f404d43596\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") "
Jul 29 18:54:22.335575 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.335436    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-uds\") pod \"8c7a1339-1b48-41f4-b75d-97f404d43596\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") "
Jul 29 18:54:22.335575 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.335467    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-kserve-provision-location\") pod \"8c7a1339-1b48-41f4-b75d-97f404d43596\" (UID: \"8c7a1339-1b48-41f4-b75d-97f404d43596\") "
Jul 29 18:54:22.335805 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.335727    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "8c7a1339-1b48-41f4-b75d-97f404d43596" (UID: "8c7a1339-1b48-41f4-b75d-97f404d43596"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:54:22.335863 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.335798    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "8c7a1339-1b48-41f4-b75d-97f404d43596" (UID: "8c7a1339-1b48-41f4-b75d-97f404d43596"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:54:22.335863 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.335808    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "8c7a1339-1b48-41f4-b75d-97f404d43596" (UID: "8c7a1339-1b48-41f4-b75d-97f404d43596"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:54:22.336306 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.336284    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "8c7a1339-1b48-41f4-b75d-97f404d43596" (UID: "8c7a1339-1b48-41f4-b75d-97f404d43596"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:54:22.337559 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.337524    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c7a1339-1b48-41f4-b75d-97f404d43596-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8c7a1339-1b48-41f4-b75d-97f404d43596" (UID: "8c7a1339-1b48-41f4-b75d-97f404d43596"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:54:22.337559 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.337544    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c7a1339-1b48-41f4-b75d-97f404d43596-kube-api-access-g6fh7" (OuterVolumeSpecName: "kube-api-access-g6fh7") pod "8c7a1339-1b48-41f4-b75d-97f404d43596" (UID: "8c7a1339-1b48-41f4-b75d-97f404d43596"). InnerVolumeSpecName "kube-api-access-g6fh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:54:22.436523 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.436464    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:54:22.436710 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.436545    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6fh7\" (UniqueName: \"kubernetes.io/projected/8c7a1339-1b48-41f4-b75d-97f404d43596-kube-api-access-g6fh7\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:54:22.436710 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.436557    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-uds\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:54:22.436710 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.436568    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-kserve-provision-location\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:54:22.436710 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.436578    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c7a1339-1b48-41f4-b75d-97f404d43596-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:54:22.436710 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.436587    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c7a1339-1b48-41f4-b75d-97f404d43596-tokenizer-tmp\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:54:22.613288 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.613253    2579 generic.go:356] "Generic (PLEG): container finished" podID="8c7a1339-1b48-41f4-b75d-97f404d43596" containerID="4c591ead22cae6e5b4488ff86e5053c699d74867767cc5c4c4b7750e175beea5" exitCode=0
Jul 29 18:54:22.613466 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.613313    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"
Jul 29 18:54:22.613466 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.613338    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz" event={"ID":"8c7a1339-1b48-41f4-b75d-97f404d43596","Type":"ContainerDied","Data":"4c591ead22cae6e5b4488ff86e5053c699d74867767cc5c4c4b7750e175beea5"}
Jul 29 18:54:22.613466 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.613396    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz" event={"ID":"8c7a1339-1b48-41f4-b75d-97f404d43596","Type":"ContainerDied","Data":"db3b501c20963403799cb0c8762c9c478af317e870f157a77d6d3d4d5714043a"}
Jul 29 18:54:22.613466 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.613422    2579 scope.go:117] "RemoveContainer" containerID="4c591ead22cae6e5b4488ff86e5053c699d74867767cc5c4c4b7750e175beea5"
Jul 29 18:54:22.622548 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.622526    2579 scope.go:117] "RemoveContainer" containerID="faf1710999ae1a05f25465ade721308a144578a5a0f36f3f5e73d758ef2c9645"
Jul 29 18:54:22.631555 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.631527    2579 scope.go:117] "RemoveContainer" containerID="a0720cca34b0838a80bbfa16537ce48a69eb0d23dfcf8043d9faed902836aae9"
Jul 29 18:54:22.638349 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.638316    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"]
Jul 29 18:54:22.641715 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.641694    2579 scope.go:117] "RemoveContainer" containerID="4c591ead22cae6e5b4488ff86e5053c699d74867767cc5c4c4b7750e175beea5"
Jul 29 18:54:22.642071 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:54:22.642047    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c591ead22cae6e5b4488ff86e5053c699d74867767cc5c4c4b7750e175beea5\": container with ID starting with 4c591ead22cae6e5b4488ff86e5053c699d74867767cc5c4c4b7750e175beea5 not found: ID does not exist" containerID="4c591ead22cae6e5b4488ff86e5053c699d74867767cc5c4c4b7750e175beea5"
Jul 29 18:54:22.642200 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.642079    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c591ead22cae6e5b4488ff86e5053c699d74867767cc5c4c4b7750e175beea5"} err="failed to get container status \"4c591ead22cae6e5b4488ff86e5053c699d74867767cc5c4c4b7750e175beea5\": rpc error: code = NotFound desc = could not find container \"4c591ead22cae6e5b4488ff86e5053c699d74867767cc5c4c4b7750e175beea5\": container with ID starting with 4c591ead22cae6e5b4488ff86e5053c699d74867767cc5c4c4b7750e175beea5 not found: ID does not exist"
Jul 29 18:54:22.642200 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.642108    2579 scope.go:117] "RemoveContainer" containerID="faf1710999ae1a05f25465ade721308a144578a5a0f36f3f5e73d758ef2c9645"
Jul 29 18:54:22.642398 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:54:22.642377    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faf1710999ae1a05f25465ade721308a144578a5a0f36f3f5e73d758ef2c9645\": container with ID starting with faf1710999ae1a05f25465ade721308a144578a5a0f36f3f5e73d758ef2c9645 not found: ID does not exist" containerID="faf1710999ae1a05f25465ade721308a144578a5a0f36f3f5e73d758ef2c9645"
Jul 29 18:54:22.642473 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.642407    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faf1710999ae1a05f25465ade721308a144578a5a0f36f3f5e73d758ef2c9645"} err="failed to get container status \"faf1710999ae1a05f25465ade721308a144578a5a0f36f3f5e73d758ef2c9645\": rpc error: code = NotFound desc = could not find container \"faf1710999ae1a05f25465ade721308a144578a5a0f36f3f5e73d758ef2c9645\": container with ID starting with faf1710999ae1a05f25465ade721308a144578a5a0f36f3f5e73d758ef2c9645 not found: ID does not exist"
Jul 29 18:54:22.642473 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.642432    2579 scope.go:117] "RemoveContainer" containerID="a0720cca34b0838a80bbfa16537ce48a69eb0d23dfcf8043d9faed902836aae9"
Jul 29 18:54:22.642616 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.642574    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz"]
Jul 29 18:54:22.642715 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:54:22.642693    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0720cca34b0838a80bbfa16537ce48a69eb0d23dfcf8043d9faed902836aae9\": container with ID starting with a0720cca34b0838a80bbfa16537ce48a69eb0d23dfcf8043d9faed902836aae9 not found: ID does not exist" containerID="a0720cca34b0838a80bbfa16537ce48a69eb0d23dfcf8043d9faed902836aae9"
Jul 29 18:54:22.642808 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:22.642722    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0720cca34b0838a80bbfa16537ce48a69eb0d23dfcf8043d9faed902836aae9"} err="failed to get container status \"a0720cca34b0838a80bbfa16537ce48a69eb0d23dfcf8043d9faed902836aae9\": rpc error: code = NotFound desc = could not find container \"a0720cca34b0838a80bbfa16537ce48a69eb0d23dfcf8043d9faed902836aae9\": container with ID starting with a0720cca34b0838a80bbfa16537ce48a69eb0d23dfcf8043d9faed902836aae9 not found: ID does not exist"
Jul 29 18:54:23.070307 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:23.070256    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-7cd7c566dd-nsxjz" podUID="8c7a1339-1b48-41f4-b75d-97f404d43596" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.132.0.42:9003\" within 1s: context deadline exceeded"
Jul 29 18:54:24.585136 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:24.585095    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c7a1339-1b48-41f4-b75d-97f404d43596" path="/var/lib/kubelet/pods/8c7a1339-1b48-41f4-b75d-97f404d43596/volumes"
Jul 29 18:54:34.504754 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:34.504722    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 18:54:34.507723 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:54:34.507694    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 18:55:23.288697 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:23.288654    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"]
Jul 29 18:55:23.289204 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:23.288995    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw" podUID="708ef420-37aa-4e0b-bc7a-9fe92379b472" containerName="main" containerID="cri-o://c21c0c951b43074516fb53312031a0ce058bce1877553280d562d2fe52dfefa2" gracePeriod=30
Jul 29 18:55:23.289204 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:23.289079    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw" podUID="708ef420-37aa-4e0b-bc7a-9fe92379b472" containerName="tokenizer" containerID="cri-o://b7c0487682b7798f4dd02c67f8ae8e2d463b118a5b12f8fdcae1c851407136c5" gracePeriod=30
Jul 29 18:55:23.923034 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:23.922999    2579 generic.go:356] "Generic (PLEG): container finished" podID="708ef420-37aa-4e0b-bc7a-9fe92379b472" containerID="c21c0c951b43074516fb53312031a0ce058bce1877553280d562d2fe52dfefa2" exitCode=0
Jul 29 18:55:23.923214 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:23.923053    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw" event={"ID":"708ef420-37aa-4e0b-bc7a-9fe92379b472","Type":"ContainerDied","Data":"c21c0c951b43074516fb53312031a0ce058bce1877553280d562d2fe52dfefa2"}
Jul 29 18:55:24.753871 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.753838    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:55:24.907849 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.907742    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-kserve-provision-location\") pod \"708ef420-37aa-4e0b-bc7a-9fe92379b472\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") "
Jul 29 18:55:24.907849 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.907804    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-cache\") pod \"708ef420-37aa-4e0b-bc7a-9fe92379b472\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") "
Jul 29 18:55:24.907849 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.907838    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-tmp\") pod \"708ef420-37aa-4e0b-bc7a-9fe92379b472\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") "
Jul 29 18:55:24.907849 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.907863    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-uds\") pod \"708ef420-37aa-4e0b-bc7a-9fe92379b472\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") "
Jul 29 18:55:24.908207 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.907909    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/708ef420-37aa-4e0b-bc7a-9fe92379b472-tls-certs\") pod \"708ef420-37aa-4e0b-bc7a-9fe92379b472\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") "
Jul 29 18:55:24.908207 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.907926    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szppj\" (UniqueName: \"kubernetes.io/projected/708ef420-37aa-4e0b-bc7a-9fe92379b472-kube-api-access-szppj\") pod \"708ef420-37aa-4e0b-bc7a-9fe92379b472\" (UID: \"708ef420-37aa-4e0b-bc7a-9fe92379b472\") "
Jul 29 18:55:24.908207 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.908144    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "708ef420-37aa-4e0b-bc7a-9fe92379b472" (UID: "708ef420-37aa-4e0b-bc7a-9fe92379b472"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:55:24.908207 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.908157    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "708ef420-37aa-4e0b-bc7a-9fe92379b472" (UID: "708ef420-37aa-4e0b-bc7a-9fe92379b472"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:55:24.908371 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.908256    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "708ef420-37aa-4e0b-bc7a-9fe92379b472" (UID: "708ef420-37aa-4e0b-bc7a-9fe92379b472"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:55:24.908483 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.908462    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "708ef420-37aa-4e0b-bc7a-9fe92379b472" (UID: "708ef420-37aa-4e0b-bc7a-9fe92379b472"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:55:24.910047 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.910022    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/708ef420-37aa-4e0b-bc7a-9fe92379b472-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "708ef420-37aa-4e0b-bc7a-9fe92379b472" (UID: "708ef420-37aa-4e0b-bc7a-9fe92379b472"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:55:24.910162 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.910098    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/708ef420-37aa-4e0b-bc7a-9fe92379b472-kube-api-access-szppj" (OuterVolumeSpecName: "kube-api-access-szppj") pod "708ef420-37aa-4e0b-bc7a-9fe92379b472" (UID: "708ef420-37aa-4e0b-bc7a-9fe92379b472"). InnerVolumeSpecName "kube-api-access-szppj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:55:24.929486 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.929449    2579 generic.go:356] "Generic (PLEG): container finished" podID="708ef420-37aa-4e0b-bc7a-9fe92379b472" containerID="b7c0487682b7798f4dd02c67f8ae8e2d463b118a5b12f8fdcae1c851407136c5" exitCode=0
Jul 29 18:55:24.929667 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.929527    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"
Jul 29 18:55:24.929667 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.929541    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw" event={"ID":"708ef420-37aa-4e0b-bc7a-9fe92379b472","Type":"ContainerDied","Data":"b7c0487682b7798f4dd02c67f8ae8e2d463b118a5b12f8fdcae1c851407136c5"}
Jul 29 18:55:24.929667 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.929595    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw" event={"ID":"708ef420-37aa-4e0b-bc7a-9fe92379b472","Type":"ContainerDied","Data":"62503a4990f00ed38d5df249d6f3c580ac926aaeded12497b0623430a3679e13"}
Jul 29 18:55:24.929667 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.929619    2579 scope.go:117] "RemoveContainer" containerID="b7c0487682b7798f4dd02c67f8ae8e2d463b118a5b12f8fdcae1c851407136c5"
Jul 29 18:55:24.939373 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.939351    2579 scope.go:117] "RemoveContainer" containerID="c21c0c951b43074516fb53312031a0ce058bce1877553280d562d2fe52dfefa2"
Jul 29 18:55:24.947466 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.947444    2579 scope.go:117] "RemoveContainer" containerID="c1b5606b1733c6a3939d299c46136f8a04a1ef87c8415b360a4db28ce2a06127"
Jul 29 18:55:24.953593 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.953566    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"]
Jul 29 18:55:24.956348 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.956328    2579 scope.go:117] "RemoveContainer" containerID="b7c0487682b7798f4dd02c67f8ae8e2d463b118a5b12f8fdcae1c851407136c5"
Jul 29 18:55:24.956828 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:55:24.956783    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7c0487682b7798f4dd02c67f8ae8e2d463b118a5b12f8fdcae1c851407136c5\": container with ID starting with b7c0487682b7798f4dd02c67f8ae8e2d463b118a5b12f8fdcae1c851407136c5 not found: ID does not exist" containerID="b7c0487682b7798f4dd02c67f8ae8e2d463b118a5b12f8fdcae1c851407136c5"
Jul 29 18:55:24.956944 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.956821    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7c0487682b7798f4dd02c67f8ae8e2d463b118a5b12f8fdcae1c851407136c5"} err="failed to get container status \"b7c0487682b7798f4dd02c67f8ae8e2d463b118a5b12f8fdcae1c851407136c5\": rpc error: code = NotFound desc = could not find container \"b7c0487682b7798f4dd02c67f8ae8e2d463b118a5b12f8fdcae1c851407136c5\": container with ID starting with b7c0487682b7798f4dd02c67f8ae8e2d463b118a5b12f8fdcae1c851407136c5 not found: ID does not exist"
Jul 29 18:55:24.956944 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.956847    2579 scope.go:117] "RemoveContainer" containerID="c21c0c951b43074516fb53312031a0ce058bce1877553280d562d2fe52dfefa2"
Jul 29 18:55:24.957304 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:55:24.957258    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c21c0c951b43074516fb53312031a0ce058bce1877553280d562d2fe52dfefa2\": container with ID starting with c21c0c951b43074516fb53312031a0ce058bce1877553280d562d2fe52dfefa2 not found: ID does not exist" containerID="c21c0c951b43074516fb53312031a0ce058bce1877553280d562d2fe52dfefa2"
Jul 29 18:55:24.957413 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.957297    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c21c0c951b43074516fb53312031a0ce058bce1877553280d562d2fe52dfefa2"} err="failed to get container status \"c21c0c951b43074516fb53312031a0ce058bce1877553280d562d2fe52dfefa2\": rpc error: code = NotFound desc = could not find container \"c21c0c951b43074516fb53312031a0ce058bce1877553280d562d2fe52dfefa2\": container with ID starting with c21c0c951b43074516fb53312031a0ce058bce1877553280d562d2fe52dfefa2 not found: ID does not exist"
Jul 29 18:55:24.957413 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.957319    2579 scope.go:117] "RemoveContainer" containerID="c1b5606b1733c6a3939d299c46136f8a04a1ef87c8415b360a4db28ce2a06127"
Jul 29 18:55:24.957705 ip-10-0-137-187 kubenswrapper[2579]: E0729 18:55:24.957683    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1b5606b1733c6a3939d299c46136f8a04a1ef87c8415b360a4db28ce2a06127\": container with ID starting with c1b5606b1733c6a3939d299c46136f8a04a1ef87c8415b360a4db28ce2a06127 not found: ID does not exist" containerID="c1b5606b1733c6a3939d299c46136f8a04a1ef87c8415b360a4db28ce2a06127"
Jul 29 18:55:24.957759 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.957709    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1b5606b1733c6a3939d299c46136f8a04a1ef87c8415b360a4db28ce2a06127"} err="failed to get container status \"c1b5606b1733c6a3939d299c46136f8a04a1ef87c8415b360a4db28ce2a06127\": rpc error: code = NotFound desc = could not find container \"c1b5606b1733c6a3939d299c46136f8a04a1ef87c8415b360a4db28ce2a06127\": container with ID starting with c1b5606b1733c6a3939d299c46136f8a04a1ef87c8415b360a4db28ce2a06127 not found: ID does not exist"
Jul 29 18:55:24.959327 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:24.959305    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-95d8b89899pqw"]
Jul 29 18:55:25.008770 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:25.008737    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-tmp\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:55:25.008770 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:25.008765    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-uds\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:55:25.008770 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:25.008775    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/708ef420-37aa-4e0b-bc7a-9fe92379b472-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:55:25.009007 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:25.008785    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szppj\" (UniqueName: \"kubernetes.io/projected/708ef420-37aa-4e0b-bc7a-9fe92379b472-kube-api-access-szppj\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:55:25.009007 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:25.008794    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-kserve-provision-location\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:55:25.009007 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:25.008806    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/708ef420-37aa-4e0b-bc7a-9fe92379b472-tokenizer-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 18:55:26.585485 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:26.585402    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="708ef420-37aa-4e0b-bc7a-9fe92379b472" path="/var/lib/kubelet/pods/708ef420-37aa-4e0b-bc7a-9fe92379b472/volumes"
Jul 29 18:55:39.569325 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569287    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"]
Jul 29 18:55:39.569783 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569672    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c7a1339-1b48-41f4-b75d-97f404d43596" containerName="storage-initializer"
Jul 29 18:55:39.569783 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569685    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c7a1339-1b48-41f4-b75d-97f404d43596" containerName="storage-initializer"
Jul 29 18:55:39.569783 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569693    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c7a1339-1b48-41f4-b75d-97f404d43596" containerName="main"
Jul 29 18:55:39.569783 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569699    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c7a1339-1b48-41f4-b75d-97f404d43596" containerName="main"
Jul 29 18:55:39.569783 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569707    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c7a1339-1b48-41f4-b75d-97f404d43596" containerName="tokenizer"
Jul 29 18:55:39.569783 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569713    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c7a1339-1b48-41f4-b75d-97f404d43596" containerName="tokenizer"
Jul 29 18:55:39.569783 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569721    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="708ef420-37aa-4e0b-bc7a-9fe92379b472" containerName="main"
Jul 29 18:55:39.569783 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569726    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="708ef420-37aa-4e0b-bc7a-9fe92379b472" containerName="main"
Jul 29 18:55:39.569783 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569733    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="708ef420-37aa-4e0b-bc7a-9fe92379b472" containerName="tokenizer"
Jul 29 18:55:39.569783 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569740    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="708ef420-37aa-4e0b-bc7a-9fe92379b472" containerName="tokenizer"
Jul 29 18:55:39.569783 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569758    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="708ef420-37aa-4e0b-bc7a-9fe92379b472" containerName="storage-initializer"
Jul 29 18:55:39.569783 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569762    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="708ef420-37aa-4e0b-bc7a-9fe92379b472" containerName="storage-initializer"
Jul 29 18:55:39.570161 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569825    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c7a1339-1b48-41f4-b75d-97f404d43596" containerName="main"
Jul 29 18:55:39.570161 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569834    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="708ef420-37aa-4e0b-bc7a-9fe92379b472" containerName="main"
Jul 29 18:55:39.570161 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569841    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="708ef420-37aa-4e0b-bc7a-9fe92379b472" containerName="tokenizer"
Jul 29 18:55:39.570161 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.569847    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c7a1339-1b48-41f4-b75d-97f404d43596" containerName="tokenizer"
Jul 29 18:55:39.574283 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.574262    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.577201 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.577170    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-test-kserve-self-signed-certs\""
Jul 29 18:55:39.577201 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.577170    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 29 18:55:39.578187 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.578165    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-test-epp-sa-dockercfg-4lk69\""
Jul 29 18:55:39.578329 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.578235    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 29 18:55:39.578329 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.578257    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2cmt9\""
Jul 29 18:55:39.584211 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.584187    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"]
Jul 29 18:55:39.631422 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.631385    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr9m4\" (UniqueName: \"kubernetes.io/projected/a0722860-f179-4bad-ace1-d60da3db81b4-kube-api-access-kr9m4\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.631639 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.631457    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.631639 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.631480    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-tmp\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.631639 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.631495    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-cache\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.631639 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.631570    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.631639 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.631605    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0722860-f179-4bad-ace1-d60da3db81b4-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.732071 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.732026    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr9m4\" (UniqueName: \"kubernetes.io/projected/a0722860-f179-4bad-ace1-d60da3db81b4-kube-api-access-kr9m4\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.732268 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.732108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.732268 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.732129    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-tmp\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.732268 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.732147    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-cache\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.732268 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.732173    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.732268 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.732194    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0722860-f179-4bad-ace1-d60da3db81b4-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.732610 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.732580    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-tmp\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.732610 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.732605    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.732758 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.732660    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-cache\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.732758 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.732702    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.734837 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.734818    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0722860-f179-4bad-ace1-d60da3db81b4-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.740209 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.740188    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr9m4\" (UniqueName: \"kubernetes.io/projected/a0722860-f179-4bad-ace1-d60da3db81b4-kube-api-access-kr9m4\") pod \"router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:39.884186 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:39.884096    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:40.035314 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:40.035284    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"]
Jul 29 18:55:40.037427 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:55:40.037397    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0722860_f179_4bad_ace1_d60da3db81b4.slice/crio-1c446507c3914bfdc27b94de1ec0a4bbef4bd555dae34d6a0bf8482b4f6c69b1 WatchSource:0}: Error finding container 1c446507c3914bfdc27b94de1ec0a4bbef4bd555dae34d6a0bf8482b4f6c69b1: Status 404 returned error can't find the container with id 1c446507c3914bfdc27b94de1ec0a4bbef4bd555dae34d6a0bf8482b4f6c69b1
Jul 29 18:55:41.007803 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:41.007761    2579 generic.go:356] "Generic (PLEG): container finished" podID="a0722860-f179-4bad-ace1-d60da3db81b4" containerID="8861b6269d7be9ac51c8fa75275e2642857b7ab2f874f4a0ffc8348bcbe3ec54" exitCode=0
Jul 29 18:55:41.008191 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:41.007821    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q" event={"ID":"a0722860-f179-4bad-ace1-d60da3db81b4","Type":"ContainerDied","Data":"8861b6269d7be9ac51c8fa75275e2642857b7ab2f874f4a0ffc8348bcbe3ec54"}
Jul 29 18:55:41.008191 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:41.007900    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q" event={"ID":"a0722860-f179-4bad-ace1-d60da3db81b4","Type":"ContainerStarted","Data":"1c446507c3914bfdc27b94de1ec0a4bbef4bd555dae34d6a0bf8482b4f6c69b1"}
Jul 29 18:55:42.015334 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:42.015291    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q" event={"ID":"a0722860-f179-4bad-ace1-d60da3db81b4","Type":"ContainerStarted","Data":"5b2a3f60cb124107e3e0fc7de1c74c9b4ff995875547ea59ee3445dd2cb82ead"}
Jul 29 18:55:42.015808 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:42.015530    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:42.015808 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:42.015583    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q" event={"ID":"a0722860-f179-4bad-ace1-d60da3db81b4","Type":"ContainerStarted","Data":"f4cbea05d0e7c4fe5bf13bfc786a3ab119b90424e294b22c2fca9ed5aae62e6c"}
Jul 29 18:55:42.036486 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:42.036432    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q" podStartSLOduration=2.24040828 podStartE2EDuration="3.036413826s" totalImagesPullingTime="0s" totalInitContainerRuntime="796.005546ms" isStatefulPod=false podCreationTimestamp="2026-07-29 18:55:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:55:42.035314114 +0000 UTC m=+1267.986881190" watchObservedRunningTime="2026-07-29 18:55:42.036413826 +0000 UTC m=+1267.987980902"
Jul 29 18:55:45.341521 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.341469    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"]
Jul 29 18:55:45.345291 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.345272    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.348301 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.348279    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-pd-test-epp-sa-dockercfg-hvrvx\""
Jul 29 18:55:45.348414 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.348279    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Jul 29 18:55:45.355920 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.355893    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"]
Jul 29 18:55:45.381316 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.381279    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz5xm\" (UniqueName: \"kubernetes.io/projected/5d3aff26-d1d2-4998-a275-ab9baa4c439f-kube-api-access-qz5xm\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.381513 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.381338    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.381513 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.381367    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.381513 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.381399    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-cache\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.381513 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.381430    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-tmp\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.381513 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.381455    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.482626 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.482587    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.482840 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.482640    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.482840 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.482702    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-cache\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.482840 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.482776    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-tmp\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.482984 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.482903    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.483043 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.483005    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz5xm\" (UniqueName: \"kubernetes.io/projected/5d3aff26-d1d2-4998-a275-ab9baa4c439f-kube-api-access-qz5xm\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.483096 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.483064    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.483218 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.483198    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.483302 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.483217    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-tmp\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.483341 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.483317    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-cache\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.485360 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.485339    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.491679 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.491650    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz5xm\" (UniqueName: \"kubernetes.io/projected/5d3aff26-d1d2-4998-a275-ab9baa4c439f-kube-api-access-qz5xm\") pod \"router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.655680 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.655578    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:45.803364 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:45.803333    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"]
Jul 29 18:55:45.805146 ip-10-0-137-187 kubenswrapper[2579]: W0729 18:55:45.805105    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d3aff26_d1d2_4998_a275_ab9baa4c439f.slice/crio-521dbfaf55a8f2cc0a36bb4e8232e51cfc738c2808896921d5e6e50ebf2e2aee WatchSource:0}: Error finding container 521dbfaf55a8f2cc0a36bb4e8232e51cfc738c2808896921d5e6e50ebf2e2aee: Status 404 returned error can't find the container with id 521dbfaf55a8f2cc0a36bb4e8232e51cfc738c2808896921d5e6e50ebf2e2aee
Jul 29 18:55:46.037062 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:46.037028    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n" event={"ID":"5d3aff26-d1d2-4998-a275-ab9baa4c439f","Type":"ContainerStarted","Data":"68f7e6b1557dfc84628c34ca3baccc4eaaa327478ec2a12a7efa263e3da30fb3"}
Jul 29 18:55:46.037239 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:46.037082    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n" event={"ID":"5d3aff26-d1d2-4998-a275-ab9baa4c439f","Type":"ContainerStarted","Data":"521dbfaf55a8f2cc0a36bb4e8232e51cfc738c2808896921d5e6e50ebf2e2aee"}
Jul 29 18:55:47.052846 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:47.052787    2579 generic.go:356] "Generic (PLEG): container finished" podID="5d3aff26-d1d2-4998-a275-ab9baa4c439f" containerID="68f7e6b1557dfc84628c34ca3baccc4eaaa327478ec2a12a7efa263e3da30fb3" exitCode=0
Jul 29 18:55:47.053325 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:47.052923    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n" event={"ID":"5d3aff26-d1d2-4998-a275-ab9baa4c439f","Type":"ContainerDied","Data":"68f7e6b1557dfc84628c34ca3baccc4eaaa327478ec2a12a7efa263e3da30fb3"}
Jul 29 18:55:48.062838 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:48.062799    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n" event={"ID":"5d3aff26-d1d2-4998-a275-ab9baa4c439f","Type":"ContainerStarted","Data":"ac1ffc0aaaebe3ad1b04368ce62a9276eb927e727cfdba3523b254f9323a9042"}
Jul 29 18:55:48.063292 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:48.062864    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n" event={"ID":"5d3aff26-d1d2-4998-a275-ab9baa4c439f","Type":"ContainerStarted","Data":"a6b31d211b8c5223cac0520e4987e8ce59302cebc08f6e0808e849122fdb07b6"}
Jul 29 18:55:48.063292 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:48.062948    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:48.085497 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:48.085434    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n" podStartSLOduration=2.307123762 podStartE2EDuration="3.085416258s" totalImagesPullingTime="0s" totalInitContainerRuntime="778.292496ms" isStatefulPod=false podCreationTimestamp="2026-07-29 18:55:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:55:48.083917134 +0000 UTC m=+1274.035484240" watchObservedRunningTime="2026-07-29 18:55:48.085416258 +0000 UTC m=+1274.036983335"
Jul 29 18:55:49.884454 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:49.884412    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:49.885006 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:49.884479    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:49.887244 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:49.887214    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:50.076135 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:50.076091    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:55:55.656065 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:55.656021    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:55.656564 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:55.656093    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:55.659029 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:55.658990    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:55:56.108156 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:55:56.108123    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:56:11.081974 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:56:11.081945    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 18:56:17.113813 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:56:17.113780    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 18:59:34.536874 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:59:34.536844    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 18:59:34.540849 ip-10-0-137-187 kubenswrapper[2579]: I0729 18:59:34.540825    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 19:04:34.572436 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:04:34.572404    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 19:04:34.577313 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:04:34.577288    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 19:09:34.612665 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:09:34.612630    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 19:09:34.615123 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:09:34.613189    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 19:10:26.415445 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:26.415351    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"]
Jul 29 19:10:26.416058 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:26.415696    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q" podUID="a0722860-f179-4bad-ace1-d60da3db81b4" containerName="main" containerID="cri-o://f4cbea05d0e7c4fe5bf13bfc786a3ab119b90424e294b22c2fca9ed5aae62e6c" gracePeriod=30
Jul 29 19:10:26.416058 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:26.415748    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q" podUID="a0722860-f179-4bad-ace1-d60da3db81b4" containerName="tokenizer" containerID="cri-o://5b2a3f60cb124107e3e0fc7de1c74c9b4ff995875547ea59ee3445dd2cb82ead" gracePeriod=30
Jul 29 19:10:26.545270 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:26.545237    2579 generic.go:356] "Generic (PLEG): container finished" podID="a0722860-f179-4bad-ace1-d60da3db81b4" containerID="f4cbea05d0e7c4fe5bf13bfc786a3ab119b90424e294b22c2fca9ed5aae62e6c" exitCode=0
Jul 29 19:10:26.545442 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:26.545310    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q" event={"ID":"a0722860-f179-4bad-ace1-d60da3db81b4","Type":"ContainerDied","Data":"f4cbea05d0e7c4fe5bf13bfc786a3ab119b90424e294b22c2fca9ed5aae62e6c"}
Jul 29 19:10:27.788822 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.788787    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 19:10:27.854024 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.853928    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kr9m4\" (UniqueName: \"kubernetes.io/projected/a0722860-f179-4bad-ace1-d60da3db81b4-kube-api-access-kr9m4\") pod \"a0722860-f179-4bad-ace1-d60da3db81b4\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") "
Jul 29 19:10:27.854024 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.853964    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0722860-f179-4bad-ace1-d60da3db81b4-tls-certs\") pod \"a0722860-f179-4bad-ace1-d60da3db81b4\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") "
Jul 29 19:10:27.854024 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.854001    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-tmp\") pod \"a0722860-f179-4bad-ace1-d60da3db81b4\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") "
Jul 29 19:10:27.854577 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.854131    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-cache\") pod \"a0722860-f179-4bad-ace1-d60da3db81b4\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") "
Jul 29 19:10:27.854577 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.854202    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-kserve-provision-location\") pod \"a0722860-f179-4bad-ace1-d60da3db81b4\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") "
Jul 29 19:10:27.854577 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.854289    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-uds\") pod \"a0722860-f179-4bad-ace1-d60da3db81b4\" (UID: \"a0722860-f179-4bad-ace1-d60da3db81b4\") "
Jul 29 19:10:27.854687 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.854614    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "a0722860-f179-4bad-ace1-d60da3db81b4" (UID: "a0722860-f179-4bad-ace1-d60da3db81b4"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:10:27.854687 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.854629    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "a0722860-f179-4bad-ace1-d60da3db81b4" (UID: "a0722860-f179-4bad-ace1-d60da3db81b4"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:10:27.854761 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.854701    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "a0722860-f179-4bad-ace1-d60da3db81b4" (UID: "a0722860-f179-4bad-ace1-d60da3db81b4"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:10:27.855067 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.855045    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "a0722860-f179-4bad-ace1-d60da3db81b4" (UID: "a0722860-f179-4bad-ace1-d60da3db81b4"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:10:27.856252 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.856226    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0722860-f179-4bad-ace1-d60da3db81b4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a0722860-f179-4bad-ace1-d60da3db81b4" (UID: "a0722860-f179-4bad-ace1-d60da3db81b4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:10:27.856467 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.856441    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0722860-f179-4bad-ace1-d60da3db81b4-kube-api-access-kr9m4" (OuterVolumeSpecName: "kube-api-access-kr9m4") pod "a0722860-f179-4bad-ace1-d60da3db81b4" (UID: "a0722860-f179-4bad-ace1-d60da3db81b4"). InnerVolumeSpecName "kube-api-access-kr9m4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:10:27.955919 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.955881    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-uds\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:10:27.955919 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.955927    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kr9m4\" (UniqueName: \"kubernetes.io/projected/a0722860-f179-4bad-ace1-d60da3db81b4-kube-api-access-kr9m4\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:10:27.956134 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.955936    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0722860-f179-4bad-ace1-d60da3db81b4-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:10:27.956134 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.955945    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-tmp\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:10:27.956134 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.955953    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-tokenizer-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:10:27.956134 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:27.955961    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a0722860-f179-4bad-ace1-d60da3db81b4-kserve-provision-location\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:10:28.557422 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:28.557376    2579 generic.go:356] "Generic (PLEG): container finished" podID="a0722860-f179-4bad-ace1-d60da3db81b4" containerID="5b2a3f60cb124107e3e0fc7de1c74c9b4ff995875547ea59ee3445dd2cb82ead" exitCode=0
Jul 29 19:10:28.557626 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:28.557450    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"
Jul 29 19:10:28.557626 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:28.557469    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q" event={"ID":"a0722860-f179-4bad-ace1-d60da3db81b4","Type":"ContainerDied","Data":"5b2a3f60cb124107e3e0fc7de1c74c9b4ff995875547ea59ee3445dd2cb82ead"}
Jul 29 19:10:28.557626 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:28.557550    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q" event={"ID":"a0722860-f179-4bad-ace1-d60da3db81b4","Type":"ContainerDied","Data":"1c446507c3914bfdc27b94de1ec0a4bbef4bd555dae34d6a0bf8482b4f6c69b1"}
Jul 29 19:10:28.557626 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:28.557567    2579 scope.go:117] "RemoveContainer" containerID="5b2a3f60cb124107e3e0fc7de1c74c9b4ff995875547ea59ee3445dd2cb82ead"
Jul 29 19:10:28.567717 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:28.567690    2579 scope.go:117] "RemoveContainer" containerID="f4cbea05d0e7c4fe5bf13bfc786a3ab119b90424e294b22c2fca9ed5aae62e6c"
Jul 29 19:10:28.576570 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:28.576546    2579 scope.go:117] "RemoveContainer" containerID="8861b6269d7be9ac51c8fa75275e2642857b7ab2f874f4a0ffc8348bcbe3ec54"
Jul 29 19:10:28.586629 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:28.586598    2579 scope.go:117] "RemoveContainer" containerID="5b2a3f60cb124107e3e0fc7de1c74c9b4ff995875547ea59ee3445dd2cb82ead"
Jul 29 19:10:28.586952 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:10:28.586930    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b2a3f60cb124107e3e0fc7de1c74c9b4ff995875547ea59ee3445dd2cb82ead\": container with ID starting with 5b2a3f60cb124107e3e0fc7de1c74c9b4ff995875547ea59ee3445dd2cb82ead not found: ID does not exist" containerID="5b2a3f60cb124107e3e0fc7de1c74c9b4ff995875547ea59ee3445dd2cb82ead"
Jul 29 19:10:28.587021 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:28.586968    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b2a3f60cb124107e3e0fc7de1c74c9b4ff995875547ea59ee3445dd2cb82ead"} err="failed to get container status \"5b2a3f60cb124107e3e0fc7de1c74c9b4ff995875547ea59ee3445dd2cb82ead\": rpc error: code = NotFound desc = could not find container \"5b2a3f60cb124107e3e0fc7de1c74c9b4ff995875547ea59ee3445dd2cb82ead\": container with ID starting with 5b2a3f60cb124107e3e0fc7de1c74c9b4ff995875547ea59ee3445dd2cb82ead not found: ID does not exist"
Jul 29 19:10:28.587021 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:28.586993    2579 scope.go:117] "RemoveContainer" containerID="f4cbea05d0e7c4fe5bf13bfc786a3ab119b90424e294b22c2fca9ed5aae62e6c"
Jul 29 19:10:28.587335 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:10:28.587312    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4cbea05d0e7c4fe5bf13bfc786a3ab119b90424e294b22c2fca9ed5aae62e6c\": container with ID starting with f4cbea05d0e7c4fe5bf13bfc786a3ab119b90424e294b22c2fca9ed5aae62e6c not found: ID does not exist" containerID="f4cbea05d0e7c4fe5bf13bfc786a3ab119b90424e294b22c2fca9ed5aae62e6c"
Jul 29 19:10:28.587400 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:28.587344    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4cbea05d0e7c4fe5bf13bfc786a3ab119b90424e294b22c2fca9ed5aae62e6c"} err="failed to get container status \"f4cbea05d0e7c4fe5bf13bfc786a3ab119b90424e294b22c2fca9ed5aae62e6c\": rpc error: code = NotFound desc = could not find container \"f4cbea05d0e7c4fe5bf13bfc786a3ab119b90424e294b22c2fca9ed5aae62e6c\": container with ID starting with f4cbea05d0e7c4fe5bf13bfc786a3ab119b90424e294b22c2fca9ed5aae62e6c not found: ID does not exist"
Jul 29 19:10:28.587400 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:28.587363    2579 scope.go:117] "RemoveContainer" containerID="8861b6269d7be9ac51c8fa75275e2642857b7ab2f874f4a0ffc8348bcbe3ec54"
Jul 29 19:10:28.587637 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:10:28.587619    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8861b6269d7be9ac51c8fa75275e2642857b7ab2f874f4a0ffc8348bcbe3ec54\": container with ID starting with 8861b6269d7be9ac51c8fa75275e2642857b7ab2f874f4a0ffc8348bcbe3ec54 not found: ID does not exist" containerID="8861b6269d7be9ac51c8fa75275e2642857b7ab2f874f4a0ffc8348bcbe3ec54"
Jul 29 19:10:28.587709 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:28.587642    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8861b6269d7be9ac51c8fa75275e2642857b7ab2f874f4a0ffc8348bcbe3ec54"} err="failed to get container status \"8861b6269d7be9ac51c8fa75275e2642857b7ab2f874f4a0ffc8348bcbe3ec54\": rpc error: code = NotFound desc = could not find container \"8861b6269d7be9ac51c8fa75275e2642857b7ab2f874f4a0ffc8348bcbe3ec54\": container with ID starting with 8861b6269d7be9ac51c8fa75275e2642857b7ab2f874f4a0ffc8348bcbe3ec54 not found: ID does not exist"
Jul 29 19:10:28.588100 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:28.588081    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"]
Jul 29 19:10:28.600273 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:28.600238    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-7647548f46-sfd4q"]
Jul 29 19:10:30.585750 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:30.585712    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0722860-f179-4bad-ace1-d60da3db81b4" path="/var/lib/kubelet/pods/a0722860-f179-4bad-ace1-d60da3db81b4/volumes"
Jul 29 19:10:45.648156 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.648119    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"]
Jul 29 19:10:45.648580 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.648495    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0722860-f179-4bad-ace1-d60da3db81b4" containerName="main"
Jul 29 19:10:45.648580 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.648521    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0722860-f179-4bad-ace1-d60da3db81b4" containerName="main"
Jul 29 19:10:45.648580 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.648534    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0722860-f179-4bad-ace1-d60da3db81b4" containerName="tokenizer"
Jul 29 19:10:45.648580 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.648540    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0722860-f179-4bad-ace1-d60da3db81b4" containerName="tokenizer"
Jul 29 19:10:45.648580 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.648547    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0722860-f179-4bad-ace1-d60da3db81b4" containerName="storage-initializer"
Jul 29 19:10:45.648580 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.648553    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0722860-f179-4bad-ace1-d60da3db81b4" containerName="storage-initializer"
Jul 29 19:10:45.648786 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.648614    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0722860-f179-4bad-ace1-d60da3db81b4" containerName="main"
Jul 29 19:10:45.648786 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.648622    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0722860-f179-4bad-ace1-d60da3db81b4" containerName="tokenizer"
Jul 29 19:10:45.652010 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.651994    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.663092 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.663072    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-05aa9bba-epp-sa-dockercfg-wsd9f\""
Jul 29 19:10:45.663334 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.663313    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv8f1a6f044e8c7a4d31a250e0c4861caf-kserve-self-signed-certs\""
Jul 29 19:10:45.703129 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.703088    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"]
Jul 29 19:10:45.714368 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.714332    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d37050b-65f2-4166-b7e4-f11d829a131c-tls-certs\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.714592 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.714415    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-uds\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.714592 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.714439    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-tmp\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.714592 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.714541    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-cache\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.714775 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.714685    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4hvq\" (UniqueName: \"kubernetes.io/projected/8d37050b-65f2-4166-b7e4-f11d829a131c-kube-api-access-g4hvq\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.714775 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.714749    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-kserve-provision-location\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.816040 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.816002    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d37050b-65f2-4166-b7e4-f11d829a131c-tls-certs\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.816218 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.816050    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-uds\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.816218 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.816068    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-tmp\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.816218 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.816085    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-cache\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.816218 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.816162    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4hvq\" (UniqueName: \"kubernetes.io/projected/8d37050b-65f2-4166-b7e4-f11d829a131c-kube-api-access-g4hvq\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.816218 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.816205    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-kserve-provision-location\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.816560 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.816497    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-uds\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.816631 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.816553    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-tmp\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.816631 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.816581    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-cache\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.816631 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.816611    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-kserve-provision-location\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.818607 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.818588    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d37050b-65f2-4166-b7e4-f11d829a131c-tls-certs\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.826589 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.826569    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4hvq\" (UniqueName: \"kubernetes.io/projected/8d37050b-65f2-4166-b7e4-f11d829a131c-kube-api-access-g4hvq\") pod \"llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") " pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:45.962198 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:45.962089    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:46.120295 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:46.120259    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"]
Jul 29 19:10:46.120570 ip-10-0-137-187 kubenswrapper[2579]: W0729 19:10:46.120542    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d37050b_65f2_4166_b7e4_f11d829a131c.slice/crio-dcac6bd0df8fd9b2db14aed71cff2a2c0b1a7c0ad3ac67ed47a50e70ab63c9ec WatchSource:0}: Error finding container dcac6bd0df8fd9b2db14aed71cff2a2c0b1a7c0ad3ac67ed47a50e70ab63c9ec: Status 404 returned error can't find the container with id dcac6bd0df8fd9b2db14aed71cff2a2c0b1a7c0ad3ac67ed47a50e70ab63c9ec
Jul 29 19:10:46.647412 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:46.647375    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5" event={"ID":"8d37050b-65f2-4166-b7e4-f11d829a131c","Type":"ContainerStarted","Data":"8420ae6b5516c71780879060f6869f26107a79c3e58fd57d35081ba07078a2a4"}
Jul 29 19:10:46.647628 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:46.647434    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5" event={"ID":"8d37050b-65f2-4166-b7e4-f11d829a131c","Type":"ContainerStarted","Data":"dcac6bd0df8fd9b2db14aed71cff2a2c0b1a7c0ad3ac67ed47a50e70ab63c9ec"}
Jul 29 19:10:47.654204 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:47.654164    2579 generic.go:356] "Generic (PLEG): container finished" podID="8d37050b-65f2-4166-b7e4-f11d829a131c" containerID="8420ae6b5516c71780879060f6869f26107a79c3e58fd57d35081ba07078a2a4" exitCode=0
Jul 29 19:10:47.654642 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:47.654253    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5" event={"ID":"8d37050b-65f2-4166-b7e4-f11d829a131c","Type":"ContainerDied","Data":"8420ae6b5516c71780879060f6869f26107a79c3e58fd57d35081ba07078a2a4"}
Jul 29 19:10:48.144540 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:48.144482    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"]
Jul 29 19:10:48.144834 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:48.144777    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n" podUID="5d3aff26-d1d2-4998-a275-ab9baa4c439f" containerName="main" containerID="cri-o://a6b31d211b8c5223cac0520e4987e8ce59302cebc08f6e0808e849122fdb07b6" gracePeriod=30
Jul 29 19:10:48.144979 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:48.144943    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n" podUID="5d3aff26-d1d2-4998-a275-ab9baa4c439f" containerName="tokenizer" containerID="cri-o://ac1ffc0aaaebe3ad1b04368ce62a9276eb927e727cfdba3523b254f9323a9042" gracePeriod=30
Jul 29 19:10:48.663269 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:48.663231    2579 generic.go:356] "Generic (PLEG): container finished" podID="5d3aff26-d1d2-4998-a275-ab9baa4c439f" containerID="a6b31d211b8c5223cac0520e4987e8ce59302cebc08f6e0808e849122fdb07b6" exitCode=0
Jul 29 19:10:48.663738 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:48.663326    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n" event={"ID":"5d3aff26-d1d2-4998-a275-ab9baa4c439f","Type":"ContainerDied","Data":"a6b31d211b8c5223cac0520e4987e8ce59302cebc08f6e0808e849122fdb07b6"}
Jul 29 19:10:48.665879 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:48.665848    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5" event={"ID":"8d37050b-65f2-4166-b7e4-f11d829a131c","Type":"ContainerStarted","Data":"b5a58d28e6c86ba530a06aacfc2413336db3b7e1d391a3e4c8c5f2064f4184f4"}
Jul 29 19:10:48.666013 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:48.665912    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5" event={"ID":"8d37050b-65f2-4166-b7e4-f11d829a131c","Type":"ContainerStarted","Data":"91c1d91ae7b4a5c761a6bcca9a5d16c2d02193d3c12d0c5e096224982167f767"}
Jul 29 19:10:48.666013 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:48.665948    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:48.698387 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:48.698321    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5" podStartSLOduration=2.876597067 podStartE2EDuration="3.698298476s" totalImagesPullingTime="0s" totalInitContainerRuntime="821.701409ms" isStatefulPod=false podCreationTimestamp="2026-07-29 19:10:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:10:48.694281114 +0000 UTC m=+2174.645848206" watchObservedRunningTime="2026-07-29 19:10:48.698298476 +0000 UTC m=+2174.649865555"
Jul 29 19:10:49.515873 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.515844    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 19:10:49.651856 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.651760    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tls-certs\") pod \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") "
Jul 29 19:10:49.652014 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.651856    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-kserve-provision-location\") pod \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") "
Jul 29 19:10:49.652014 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.651883    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-cache\") pod \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") "
Jul 29 19:10:49.652014 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.651927    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qz5xm\" (UniqueName: \"kubernetes.io/projected/5d3aff26-d1d2-4998-a275-ab9baa4c439f-kube-api-access-qz5xm\") pod \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") "
Jul 29 19:10:49.652014 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.651950    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-tmp\") pod \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") "
Jul 29 19:10:49.652238 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.652032    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-uds\") pod \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\" (UID: \"5d3aff26-d1d2-4998-a275-ab9baa4c439f\") "
Jul 29 19:10:49.652238 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.652179    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "5d3aff26-d1d2-4998-a275-ab9baa4c439f" (UID: "5d3aff26-d1d2-4998-a275-ab9baa4c439f"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:10:49.652346 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.652323    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "5d3aff26-d1d2-4998-a275-ab9baa4c439f" (UID: "5d3aff26-d1d2-4998-a275-ab9baa4c439f"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:10:49.652401 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.652349    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "5d3aff26-d1d2-4998-a275-ab9baa4c439f" (UID: "5d3aff26-d1d2-4998-a275-ab9baa4c439f"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:10:49.652401 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.652393    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:10:49.652761 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.652737    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "5d3aff26-d1d2-4998-a275-ab9baa4c439f" (UID: "5d3aff26-d1d2-4998-a275-ab9baa4c439f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:10:49.654096 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.654076    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d3aff26-d1d2-4998-a275-ab9baa4c439f-kube-api-access-qz5xm" (OuterVolumeSpecName: "kube-api-access-qz5xm") pod "5d3aff26-d1d2-4998-a275-ab9baa4c439f" (UID: "5d3aff26-d1d2-4998-a275-ab9baa4c439f"). InnerVolumeSpecName "kube-api-access-qz5xm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:10:49.654171 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.654094    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5d3aff26-d1d2-4998-a275-ab9baa4c439f" (UID: "5d3aff26-d1d2-4998-a275-ab9baa4c439f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:10:49.673237 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.673204    2579 generic.go:356] "Generic (PLEG): container finished" podID="5d3aff26-d1d2-4998-a275-ab9baa4c439f" containerID="ac1ffc0aaaebe3ad1b04368ce62a9276eb927e727cfdba3523b254f9323a9042" exitCode=0
Jul 29 19:10:49.673625 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.673289    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n" event={"ID":"5d3aff26-d1d2-4998-a275-ab9baa4c439f","Type":"ContainerDied","Data":"ac1ffc0aaaebe3ad1b04368ce62a9276eb927e727cfdba3523b254f9323a9042"}
Jul 29 19:10:49.673625 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.673303    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"
Jul 29 19:10:49.673625 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.673341    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n" event={"ID":"5d3aff26-d1d2-4998-a275-ab9baa4c439f","Type":"ContainerDied","Data":"521dbfaf55a8f2cc0a36bb4e8232e51cfc738c2808896921d5e6e50ebf2e2aee"}
Jul 29 19:10:49.673625 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.673367    2579 scope.go:117] "RemoveContainer" containerID="ac1ffc0aaaebe3ad1b04368ce62a9276eb927e727cfdba3523b254f9323a9042"
Jul 29 19:10:49.682346 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.682328    2579 scope.go:117] "RemoveContainer" containerID="a6b31d211b8c5223cac0520e4987e8ce59302cebc08f6e0808e849122fdb07b6"
Jul 29 19:10:49.690634 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.690609    2579 scope.go:117] "RemoveContainer" containerID="68f7e6b1557dfc84628c34ca3baccc4eaaa327478ec2a12a7efa263e3da30fb3"
Jul 29 19:10:49.698749 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.698722    2579 scope.go:117] "RemoveContainer" containerID="ac1ffc0aaaebe3ad1b04368ce62a9276eb927e727cfdba3523b254f9323a9042"
Jul 29 19:10:49.699058 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:10:49.699039    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac1ffc0aaaebe3ad1b04368ce62a9276eb927e727cfdba3523b254f9323a9042\": container with ID starting with ac1ffc0aaaebe3ad1b04368ce62a9276eb927e727cfdba3523b254f9323a9042 not found: ID does not exist" containerID="ac1ffc0aaaebe3ad1b04368ce62a9276eb927e727cfdba3523b254f9323a9042"
Jul 29 19:10:49.699137 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.699073    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac1ffc0aaaebe3ad1b04368ce62a9276eb927e727cfdba3523b254f9323a9042"} err="failed to get container status \"ac1ffc0aaaebe3ad1b04368ce62a9276eb927e727cfdba3523b254f9323a9042\": rpc error: code = NotFound desc = could not find container \"ac1ffc0aaaebe3ad1b04368ce62a9276eb927e727cfdba3523b254f9323a9042\": container with ID starting with ac1ffc0aaaebe3ad1b04368ce62a9276eb927e727cfdba3523b254f9323a9042 not found: ID does not exist"
Jul 29 19:10:49.699137 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.699099    2579 scope.go:117] "RemoveContainer" containerID="a6b31d211b8c5223cac0520e4987e8ce59302cebc08f6e0808e849122fdb07b6"
Jul 29 19:10:49.699433 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:10:49.699415    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6b31d211b8c5223cac0520e4987e8ce59302cebc08f6e0808e849122fdb07b6\": container with ID starting with a6b31d211b8c5223cac0520e4987e8ce59302cebc08f6e0808e849122fdb07b6 not found: ID does not exist" containerID="a6b31d211b8c5223cac0520e4987e8ce59302cebc08f6e0808e849122fdb07b6"
Jul 29 19:10:49.699597 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.699452    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6b31d211b8c5223cac0520e4987e8ce59302cebc08f6e0808e849122fdb07b6"} err="failed to get container status \"a6b31d211b8c5223cac0520e4987e8ce59302cebc08f6e0808e849122fdb07b6\": rpc error: code = NotFound desc = could not find container \"a6b31d211b8c5223cac0520e4987e8ce59302cebc08f6e0808e849122fdb07b6\": container with ID starting with a6b31d211b8c5223cac0520e4987e8ce59302cebc08f6e0808e849122fdb07b6 not found: ID does not exist"
Jul 29 19:10:49.699597 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.699474    2579 scope.go:117] "RemoveContainer" containerID="68f7e6b1557dfc84628c34ca3baccc4eaaa327478ec2a12a7efa263e3da30fb3"
Jul 29 19:10:49.699787 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:10:49.699764    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68f7e6b1557dfc84628c34ca3baccc4eaaa327478ec2a12a7efa263e3da30fb3\": container with ID starting with 68f7e6b1557dfc84628c34ca3baccc4eaaa327478ec2a12a7efa263e3da30fb3 not found: ID does not exist" containerID="68f7e6b1557dfc84628c34ca3baccc4eaaa327478ec2a12a7efa263e3da30fb3"
Jul 29 19:10:49.699831 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.699794    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68f7e6b1557dfc84628c34ca3baccc4eaaa327478ec2a12a7efa263e3da30fb3"} err="failed to get container status \"68f7e6b1557dfc84628c34ca3baccc4eaaa327478ec2a12a7efa263e3da30fb3\": rpc error: code = NotFound desc = could not find container \"68f7e6b1557dfc84628c34ca3baccc4eaaa327478ec2a12a7efa263e3da30fb3\": container with ID starting with 68f7e6b1557dfc84628c34ca3baccc4eaaa327478ec2a12a7efa263e3da30fb3 not found: ID does not exist"
Jul 29 19:10:49.727859 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.727825    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"]
Jul 29 19:10:49.735082 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.735053    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-67ddc9546hqz9n"]
Jul 29 19:10:49.753628 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.753604    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-uds\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:10:49.753628 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.753629    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:10:49.753782 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.753640    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-kserve-provision-location\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:10:49.753782 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.753650    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz5xm\" (UniqueName: \"kubernetes.io/projected/5d3aff26-d1d2-4998-a275-ab9baa4c439f-kube-api-access-qz5xm\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:10:49.753782 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:49.753660    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d3aff26-d1d2-4998-a275-ab9baa4c439f-tokenizer-tmp\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:10:50.587646 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:50.587614    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d3aff26-d1d2-4998-a275-ab9baa4c439f" path="/var/lib/kubelet/pods/5d3aff26-d1d2-4998-a275-ab9baa4c439f/volumes"
Jul 29 19:10:55.963305 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:55.963266    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:55.963751 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:55.963359    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:55.966203 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:55.966173    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:10:56.716595 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:10:56.716566    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:11:17.724624 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:11:17.724592    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:14:00.323806 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:00.323771    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"]
Jul 29 19:14:00.324325 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:00.324014    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5" podUID="8d37050b-65f2-4166-b7e4-f11d829a131c" containerName="main" containerID="cri-o://91c1d91ae7b4a5c761a6bcca9a5d16c2d02193d3c12d0c5e096224982167f767" gracePeriod=30
Jul 29 19:14:00.324325 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:00.324075    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5" podUID="8d37050b-65f2-4166-b7e4-f11d829a131c" containerName="tokenizer" containerID="cri-o://b5a58d28e6c86ba530a06aacfc2413336db3b7e1d391a3e4c8c5f2064f4184f4" gracePeriod=30
Jul 29 19:14:00.671888 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:00.671793    2579 generic.go:356] "Generic (PLEG): container finished" podID="8d37050b-65f2-4166-b7e4-f11d829a131c" containerID="91c1d91ae7b4a5c761a6bcca9a5d16c2d02193d3c12d0c5e096224982167f767" exitCode=0
Jul 29 19:14:00.671888 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:00.671839    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5" event={"ID":"8d37050b-65f2-4166-b7e4-f11d829a131c","Type":"ContainerDied","Data":"91c1d91ae7b4a5c761a6bcca9a5d16c2d02193d3c12d0c5e096224982167f767"}
Jul 29 19:14:01.680940 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.680896    2579 generic.go:356] "Generic (PLEG): container finished" podID="8d37050b-65f2-4166-b7e4-f11d829a131c" containerID="b5a58d28e6c86ba530a06aacfc2413336db3b7e1d391a3e4c8c5f2064f4184f4" exitCode=0
Jul 29 19:14:01.681345 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.680971    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5" event={"ID":"8d37050b-65f2-4166-b7e4-f11d829a131c","Type":"ContainerDied","Data":"b5a58d28e6c86ba530a06aacfc2413336db3b7e1d391a3e4c8c5f2064f4184f4"}
Jul 29 19:14:01.784397 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.784363    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:14:01.910071 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.909972    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-tmp\") pod \"8d37050b-65f2-4166-b7e4-f11d829a131c\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") "
Jul 29 19:14:01.910071 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.910017    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4hvq\" (UniqueName: \"kubernetes.io/projected/8d37050b-65f2-4166-b7e4-f11d829a131c-kube-api-access-g4hvq\") pod \"8d37050b-65f2-4166-b7e4-f11d829a131c\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") "
Jul 29 19:14:01.910071 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.910049    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-kserve-provision-location\") pod \"8d37050b-65f2-4166-b7e4-f11d829a131c\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") "
Jul 29 19:14:01.910322 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.910095    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-cache\") pod \"8d37050b-65f2-4166-b7e4-f11d829a131c\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") "
Jul 29 19:14:01.910322 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.910138    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d37050b-65f2-4166-b7e4-f11d829a131c-tls-certs\") pod \"8d37050b-65f2-4166-b7e4-f11d829a131c\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") "
Jul 29 19:14:01.910322 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.910221    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-uds\") pod \"8d37050b-65f2-4166-b7e4-f11d829a131c\" (UID: \"8d37050b-65f2-4166-b7e4-f11d829a131c\") "
Jul 29 19:14:01.910490 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.910395    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "8d37050b-65f2-4166-b7e4-f11d829a131c" (UID: "8d37050b-65f2-4166-b7e4-f11d829a131c"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:14:01.910490 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.910406    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "8d37050b-65f2-4166-b7e4-f11d829a131c" (UID: "8d37050b-65f2-4166-b7e4-f11d829a131c"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:14:01.910638 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.910588    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:14:01.910638 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.910598    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "8d37050b-65f2-4166-b7e4-f11d829a131c" (UID: "8d37050b-65f2-4166-b7e4-f11d829a131c"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:14:01.910638 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.910611    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-tmp\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:14:01.910894 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.910869    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "8d37050b-65f2-4166-b7e4-f11d829a131c" (UID: "8d37050b-65f2-4166-b7e4-f11d829a131c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:14:01.912311 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.912279    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d37050b-65f2-4166-b7e4-f11d829a131c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8d37050b-65f2-4166-b7e4-f11d829a131c" (UID: "8d37050b-65f2-4166-b7e4-f11d829a131c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:14:01.912383 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:01.912344    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d37050b-65f2-4166-b7e4-f11d829a131c-kube-api-access-g4hvq" (OuterVolumeSpecName: "kube-api-access-g4hvq") pod "8d37050b-65f2-4166-b7e4-f11d829a131c" (UID: "8d37050b-65f2-4166-b7e4-f11d829a131c"). InnerVolumeSpecName "kube-api-access-g4hvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:14:02.011077 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:02.011032    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4hvq\" (UniqueName: \"kubernetes.io/projected/8d37050b-65f2-4166-b7e4-f11d829a131c-kube-api-access-g4hvq\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:14:02.011077 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:02.011067    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-kserve-provision-location\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:14:02.011077 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:02.011077    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d37050b-65f2-4166-b7e4-f11d829a131c-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:14:02.011077 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:02.011086    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/8d37050b-65f2-4166-b7e4-f11d829a131c-tokenizer-uds\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:14:02.688030 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:02.687997    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"
Jul 29 19:14:02.688560 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:02.688005    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5" event={"ID":"8d37050b-65f2-4166-b7e4-f11d829a131c","Type":"ContainerDied","Data":"dcac6bd0df8fd9b2db14aed71cff2a2c0b1a7c0ad3ac67ed47a50e70ab63c9ec"}
Jul 29 19:14:02.688560 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:02.688132    2579 scope.go:117] "RemoveContainer" containerID="b5a58d28e6c86ba530a06aacfc2413336db3b7e1d391a3e4c8c5f2064f4184f4"
Jul 29 19:14:02.697461 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:02.697437    2579 scope.go:117] "RemoveContainer" containerID="91c1d91ae7b4a5c761a6bcca9a5d16c2d02193d3c12d0c5e096224982167f767"
Jul 29 19:14:02.706485 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:02.706375    2579 scope.go:117] "RemoveContainer" containerID="8420ae6b5516c71780879060f6869f26107a79c3e58fd57d35081ba07078a2a4"
Jul 29 19:14:02.708433 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:02.708408    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"]
Jul 29 19:14:02.712659 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:02.712625    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-sche877n5"]
Jul 29 19:14:04.586935 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:04.586898    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d37050b-65f2-4166-b7e4-f11d829a131c" path="/var/lib/kubelet/pods/8d37050b-65f2-4166-b7e4-f11d829a131c/volumes"
Jul 29 19:14:13.474999 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.474966    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"]
Jul 29 19:14:13.475397 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.475368    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3aff26-d1d2-4998-a275-ab9baa4c439f" containerName="tokenizer"
Jul 29 19:14:13.475397 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.475378    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3aff26-d1d2-4998-a275-ab9baa4c439f" containerName="tokenizer"
Jul 29 19:14:13.475397 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.475396    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3aff26-d1d2-4998-a275-ab9baa4c439f" containerName="main"
Jul 29 19:14:13.475528 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.475402    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3aff26-d1d2-4998-a275-ab9baa4c439f" containerName="main"
Jul 29 19:14:13.475528 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.475409    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d37050b-65f2-4166-b7e4-f11d829a131c" containerName="storage-initializer"
Jul 29 19:14:13.475528 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.475414    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d37050b-65f2-4166-b7e4-f11d829a131c" containerName="storage-initializer"
Jul 29 19:14:13.475528 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.475423    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3aff26-d1d2-4998-a275-ab9baa4c439f" containerName="storage-initializer"
Jul 29 19:14:13.475528 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.475428    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3aff26-d1d2-4998-a275-ab9baa4c439f" containerName="storage-initializer"
Jul 29 19:14:13.475528 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.475434    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d37050b-65f2-4166-b7e4-f11d829a131c" containerName="tokenizer"
Jul 29 19:14:13.475528 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.475440    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d37050b-65f2-4166-b7e4-f11d829a131c" containerName="tokenizer"
Jul 29 19:14:13.475528 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.475453    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d37050b-65f2-4166-b7e4-f11d829a131c" containerName="main"
Jul 29 19:14:13.475528 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.475457    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d37050b-65f2-4166-b7e4-f11d829a131c" containerName="main"
Jul 29 19:14:13.475528 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.475531    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d3aff26-d1d2-4998-a275-ab9baa4c439f" containerName="tokenizer"
Jul 29 19:14:13.475867 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.475541    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d37050b-65f2-4166-b7e4-f11d829a131c" containerName="tokenizer"
Jul 29 19:14:13.475867 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.475549    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d3aff26-d1d2-4998-a275-ab9baa4c439f" containerName="main"
Jul 29 19:14:13.475867 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.475558    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d37050b-65f2-4166-b7e4-f11d829a131c" containerName="main"
Jul 29 19:14:13.478816 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.478797    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.481693 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.481669    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-pd-test-epp-sa-dockercfg-sqtcn\""
Jul 29 19:14:13.482772 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.482753    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 29 19:14:13.482862 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.482795    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Jul 29 19:14:13.482862 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.482817    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2cmt9\""
Jul 29 19:14:13.482982 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.482866    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 29 19:14:13.490978 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.490949    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"]
Jul 29 19:14:13.623889 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.623847    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/673008f0-aae4-4afb-8840-717cf3572a1f-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.623889 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.623893    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wqf7\" (UniqueName: \"kubernetes.io/projected/673008f0-aae4-4afb-8840-717cf3572a1f-kube-api-access-9wqf7\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.624106 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.623983    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.624106 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.624034    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-uds\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.624106 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.624058    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-cache\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.624208 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.624106    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-tmp\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.725578 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.725480    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.725578 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.725576    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-uds\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.725790 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.725600    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-cache\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.725790 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.725622    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-tmp\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.725790 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.725700    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/673008f0-aae4-4afb-8840-717cf3572a1f-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.725790 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.725730    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wqf7\" (UniqueName: \"kubernetes.io/projected/673008f0-aae4-4afb-8840-717cf3572a1f-kube-api-access-9wqf7\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.725959 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.725934    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.726003 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.725986    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-cache\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.726058 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.726037    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-uds\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.726058 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.726050    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-tmp\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.728230 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.728204    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/673008f0-aae4-4afb-8840-717cf3572a1f-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.734522 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.734478    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wqf7\" (UniqueName: \"kubernetes.io/projected/673008f0-aae4-4afb-8840-717cf3572a1f-kube-api-access-9wqf7\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.789594 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.789549    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:13.949925 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:13.949892    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"]
Jul 29 19:14:13.951080 ip-10-0-137-187 kubenswrapper[2579]: W0729 19:14:13.951050    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod673008f0_aae4_4afb_8840_717cf3572a1f.slice/crio-1b45bc419d38d5f4523a66d217c5b89e04697317227d72b44445c59455f52d42 WatchSource:0}: Error finding container 1b45bc419d38d5f4523a66d217c5b89e04697317227d72b44445c59455f52d42: Status 404 returned error can't find the container with id 1b45bc419d38d5f4523a66d217c5b89e04697317227d72b44445c59455f52d42
Jul 29 19:14:14.750412 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:14.750366    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4" event={"ID":"673008f0-aae4-4afb-8840-717cf3572a1f","Type":"ContainerStarted","Data":"187174b6e557bbbfbe3d4ded5c84a8754dabdd76533d5fc1a4678bcfabb87827"}
Jul 29 19:14:14.750862 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:14.750432    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4" event={"ID":"673008f0-aae4-4afb-8840-717cf3572a1f","Type":"ContainerStarted","Data":"1b45bc419d38d5f4523a66d217c5b89e04697317227d72b44445c59455f52d42"}
Jul 29 19:14:15.756415 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:15.756380    2579 generic.go:356] "Generic (PLEG): container finished" podID="673008f0-aae4-4afb-8840-717cf3572a1f" containerID="187174b6e557bbbfbe3d4ded5c84a8754dabdd76533d5fc1a4678bcfabb87827" exitCode=0
Jul 29 19:14:15.756867 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:15.756463    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4" event={"ID":"673008f0-aae4-4afb-8840-717cf3572a1f","Type":"ContainerDied","Data":"187174b6e557bbbfbe3d4ded5c84a8754dabdd76533d5fc1a4678bcfabb87827"}
Jul 29 19:14:16.765394 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:16.765356    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4" event={"ID":"673008f0-aae4-4afb-8840-717cf3572a1f","Type":"ContainerStarted","Data":"828a24641a0aa724403f9d8f0c864a4987e22c69bcb0ac1e7edbbe27825c681a"}
Jul 29 19:14:16.765837 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:16.765424    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4" event={"ID":"673008f0-aae4-4afb-8840-717cf3572a1f","Type":"ContainerStarted","Data":"5fff070dc75f865e370f3b1f549124cd9ec17b9714cdc109a7ff703f8cb007b7"}
Jul 29 19:14:16.765837 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:16.765528    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:16.788203 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:16.788148    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4" podStartSLOduration=2.958383557 podStartE2EDuration="3.788128508s" totalImagesPullingTime="0s" totalInitContainerRuntime="829.744951ms" isStatefulPod=false podCreationTimestamp="2026-07-29 19:14:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:14:16.786884189 +0000 UTC m=+2382.738451266" watchObservedRunningTime="2026-07-29 19:14:16.788128508 +0000 UTC m=+2382.739695585"
Jul 29 19:14:23.790390 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:23.790350    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:23.790857 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:23.790418    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:23.793175 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:23.793145    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:23.802859 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:23.802831    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:14:34.643959 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:34.643839    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 19:14:34.647184 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:34.647161    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 19:14:54.807656 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:14:54.807624    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:15:07.582634 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.582593    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"]
Jul 29 19:15:07.586370 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.586348    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.591482 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.591453    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-f312f5ec-epp-sa-dockercfg-5rsbj\""
Jul 29 19:15:07.591699 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.591600    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv4e643bc258191ffc517a31cd1d0ddd27-kserve-self-signed-certs\""
Jul 29 19:15:07.601534 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.601481    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"]
Jul 29 19:15:07.623617 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.623571    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-uds\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.623617 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.623621    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f29a06ed-7c34-42a3-bce7-216ba08887cb-tls-certs\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.623852 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.623673    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-cache\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.623852 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.623719    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlvv5\" (UniqueName: \"kubernetes.io/projected/f29a06ed-7c34-42a3-bce7-216ba08887cb-kube-api-access-rlvv5\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.623852 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.623829    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-kserve-provision-location\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.623963 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.623868    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-tmp\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.724906 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.724856    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-cache\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.724906 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.724908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlvv5\" (UniqueName: \"kubernetes.io/projected/f29a06ed-7c34-42a3-bce7-216ba08887cb-kube-api-access-rlvv5\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.725171 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.724958    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-kserve-provision-location\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.725171 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.724984    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-tmp\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.725171 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.725135    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-uds\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.725622 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.725595    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-kserve-provision-location\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.725772 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.725180    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f29a06ed-7c34-42a3-bce7-216ba08887cb-tls-certs\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.725772 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.725682    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-cache\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.725772 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.725702    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-tmp\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.725945 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.725877    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-uds\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.731426 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.729131    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f29a06ed-7c34-42a3-bce7-216ba08887cb-tls-certs\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.736096 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.736059    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlvv5\" (UniqueName: \"kubernetes.io/projected/f29a06ed-7c34-42a3-bce7-216ba08887cb-kube-api-access-rlvv5\") pod \"llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") " pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:07.898954 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:07.898847    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:08.075435 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:08.075404    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"]
Jul 29 19:15:08.077592 ip-10-0-137-187 kubenswrapper[2579]: W0729 19:15:08.077562    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf29a06ed_7c34_42a3_bce7_216ba08887cb.slice/crio-6072d1e57dbb5a11cafdc0eb569cc8b6ce0aff027a96dfa61386e4ebab110db6 WatchSource:0}: Error finding container 6072d1e57dbb5a11cafdc0eb569cc8b6ce0aff027a96dfa61386e4ebab110db6: Status 404 returned error can't find the container with id 6072d1e57dbb5a11cafdc0eb569cc8b6ce0aff027a96dfa61386e4ebab110db6
Jul 29 19:15:09.035060 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:09.035015    2579 generic.go:356] "Generic (PLEG): container finished" podID="f29a06ed-7c34-42a3-bce7-216ba08887cb" containerID="7d0f2f72fbe95253eafed78ac48a99442e77fc4d9300d69e7aeab2509b1383f5" exitCode=0
Jul 29 19:15:09.035470 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:09.035077    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk" event={"ID":"f29a06ed-7c34-42a3-bce7-216ba08887cb","Type":"ContainerDied","Data":"7d0f2f72fbe95253eafed78ac48a99442e77fc4d9300d69e7aeab2509b1383f5"}
Jul 29 19:15:09.035470 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:09.035127    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk" event={"ID":"f29a06ed-7c34-42a3-bce7-216ba08887cb","Type":"ContainerStarted","Data":"6072d1e57dbb5a11cafdc0eb569cc8b6ce0aff027a96dfa61386e4ebab110db6"}
Jul 29 19:15:10.042407 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:10.042364    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk" event={"ID":"f29a06ed-7c34-42a3-bce7-216ba08887cb","Type":"ContainerStarted","Data":"23e744029bfbf7638bac76941370a1309ad935d535be76b3ff7b3987ccfc2b50"}
Jul 29 19:15:10.042859 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:10.042432    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk" event={"ID":"f29a06ed-7c34-42a3-bce7-216ba08887cb","Type":"ContainerStarted","Data":"43d74b9fcab46b4d8645e0eddc4eca3fa1cc0d44abc230a7cea95cf8783eb424"}
Jul 29 19:15:10.042859 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:10.042541    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:10.066867 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:10.066805    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk" podStartSLOduration=2.205261259 podStartE2EDuration="3.066786181s" totalImagesPullingTime="0s" totalInitContainerRuntime="861.524922ms" isStatefulPod=false podCreationTimestamp="2026-07-29 19:15:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:15:10.065374932 +0000 UTC m=+2436.016942009" watchObservedRunningTime="2026-07-29 19:15:10.066786181 +0000 UTC m=+2436.018353258"
Jul 29 19:15:17.899259 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:17.899214    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:17.899736 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:17.899297    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:17.902041 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:17.902012    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:18.090486 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:18.090444    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:15:39.096322 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:15:39.096291    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:17:11.842133 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:11.842093    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"]
Jul 29 19:17:11.842726 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:11.842480    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4" podUID="673008f0-aae4-4afb-8840-717cf3572a1f" containerName="tokenizer" containerID="cri-o://828a24641a0aa724403f9d8f0c864a4987e22c69bcb0ac1e7edbbe27825c681a" gracePeriod=30
Jul 29 19:17:11.842726 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:11.842712    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4" podUID="673008f0-aae4-4afb-8840-717cf3572a1f" containerName="main" containerID="cri-o://5fff070dc75f865e370f3b1f549124cd9ec17b9714cdc109a7ff703f8cb007b7" gracePeriod=30
Jul 29 19:17:12.660065 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:12.660028    2579 generic.go:356] "Generic (PLEG): container finished" podID="673008f0-aae4-4afb-8840-717cf3572a1f" containerID="5fff070dc75f865e370f3b1f549124cd9ec17b9714cdc109a7ff703f8cb007b7" exitCode=0
Jul 29 19:17:12.660251 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:12.660099    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4" event={"ID":"673008f0-aae4-4afb-8840-717cf3572a1f","Type":"ContainerDied","Data":"5fff070dc75f865e370f3b1f549124cd9ec17b9714cdc109a7ff703f8cb007b7"}
Jul 29 19:17:13.219519 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.219483    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:17:13.398289 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.398170    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-cache\") pod \"673008f0-aae4-4afb-8840-717cf3572a1f\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") "
Jul 29 19:17:13.398289 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.398248    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wqf7\" (UniqueName: \"kubernetes.io/projected/673008f0-aae4-4afb-8840-717cf3572a1f-kube-api-access-9wqf7\") pod \"673008f0-aae4-4afb-8840-717cf3572a1f\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") "
Jul 29 19:17:13.398289 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.398268    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-uds\") pod \"673008f0-aae4-4afb-8840-717cf3572a1f\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") "
Jul 29 19:17:13.398623 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.398322    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-kserve-provision-location\") pod \"673008f0-aae4-4afb-8840-717cf3572a1f\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") "
Jul 29 19:17:13.398623 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.398348    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/673008f0-aae4-4afb-8840-717cf3572a1f-tls-certs\") pod \"673008f0-aae4-4afb-8840-717cf3572a1f\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") "
Jul 29 19:17:13.398623 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.398382    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-tmp\") pod \"673008f0-aae4-4afb-8840-717cf3572a1f\" (UID: \"673008f0-aae4-4afb-8840-717cf3572a1f\") "
Jul 29 19:17:13.398623 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.398530    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "673008f0-aae4-4afb-8840-717cf3572a1f" (UID: "673008f0-aae4-4afb-8840-717cf3572a1f"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:17:13.398623 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.398537    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "673008f0-aae4-4afb-8840-717cf3572a1f" (UID: "673008f0-aae4-4afb-8840-717cf3572a1f"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:17:13.398908 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.398744    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:17:13.398908 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.398766    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-uds\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:17:13.398908 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.398812    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "673008f0-aae4-4afb-8840-717cf3572a1f" (UID: "673008f0-aae4-4afb-8840-717cf3572a1f"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:17:13.399118 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.399095    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "673008f0-aae4-4afb-8840-717cf3572a1f" (UID: "673008f0-aae4-4afb-8840-717cf3572a1f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:17:13.400495 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.400470    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/673008f0-aae4-4afb-8840-717cf3572a1f-kube-api-access-9wqf7" (OuterVolumeSpecName: "kube-api-access-9wqf7") pod "673008f0-aae4-4afb-8840-717cf3572a1f" (UID: "673008f0-aae4-4afb-8840-717cf3572a1f"). InnerVolumeSpecName "kube-api-access-9wqf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:17:13.400623 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.400579    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/673008f0-aae4-4afb-8840-717cf3572a1f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "673008f0-aae4-4afb-8840-717cf3572a1f" (UID: "673008f0-aae4-4afb-8840-717cf3572a1f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:17:13.500087 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.500046    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wqf7\" (UniqueName: \"kubernetes.io/projected/673008f0-aae4-4afb-8840-717cf3572a1f-kube-api-access-9wqf7\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:17:13.500087 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.500076    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-kserve-provision-location\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:17:13.500087 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.500086    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/673008f0-aae4-4afb-8840-717cf3572a1f-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:17:13.500087 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.500096    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/673008f0-aae4-4afb-8840-717cf3572a1f-tokenizer-tmp\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:17:13.667272 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.667185    2579 generic.go:356] "Generic (PLEG): container finished" podID="673008f0-aae4-4afb-8840-717cf3572a1f" containerID="828a24641a0aa724403f9d8f0c864a4987e22c69bcb0ac1e7edbbe27825c681a" exitCode=0
Jul 29 19:17:13.667272 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.667233    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"
Jul 29 19:17:13.667272 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.667258    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4" event={"ID":"673008f0-aae4-4afb-8840-717cf3572a1f","Type":"ContainerDied","Data":"828a24641a0aa724403f9d8f0c864a4987e22c69bcb0ac1e7edbbe27825c681a"}
Jul 29 19:17:13.667486 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.667321    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4" event={"ID":"673008f0-aae4-4afb-8840-717cf3572a1f","Type":"ContainerDied","Data":"1b45bc419d38d5f4523a66d217c5b89e04697317227d72b44445c59455f52d42"}
Jul 29 19:17:13.667486 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.667354    2579 scope.go:117] "RemoveContainer" containerID="828a24641a0aa724403f9d8f0c864a4987e22c69bcb0ac1e7edbbe27825c681a"
Jul 29 19:17:13.684320 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.684298    2579 scope.go:117] "RemoveContainer" containerID="5fff070dc75f865e370f3b1f549124cd9ec17b9714cdc109a7ff703f8cb007b7"
Jul 29 19:17:13.693573 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.693547    2579 scope.go:117] "RemoveContainer" containerID="187174b6e557bbbfbe3d4ded5c84a8754dabdd76533d5fc1a4678bcfabb87827"
Jul 29 19:17:13.695282 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.695258    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"]
Jul 29 19:17:13.700006 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.699976    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-74f6bkkbb4"]
Jul 29 19:17:13.703390 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.703371    2579 scope.go:117] "RemoveContainer" containerID="828a24641a0aa724403f9d8f0c864a4987e22c69bcb0ac1e7edbbe27825c681a"
Jul 29 19:17:13.703747 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:17:13.703728    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"828a24641a0aa724403f9d8f0c864a4987e22c69bcb0ac1e7edbbe27825c681a\": container with ID starting with 828a24641a0aa724403f9d8f0c864a4987e22c69bcb0ac1e7edbbe27825c681a not found: ID does not exist" containerID="828a24641a0aa724403f9d8f0c864a4987e22c69bcb0ac1e7edbbe27825c681a"
Jul 29 19:17:13.703848 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.703759    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"828a24641a0aa724403f9d8f0c864a4987e22c69bcb0ac1e7edbbe27825c681a"} err="failed to get container status \"828a24641a0aa724403f9d8f0c864a4987e22c69bcb0ac1e7edbbe27825c681a\": rpc error: code = NotFound desc = could not find container \"828a24641a0aa724403f9d8f0c864a4987e22c69bcb0ac1e7edbbe27825c681a\": container with ID starting with 828a24641a0aa724403f9d8f0c864a4987e22c69bcb0ac1e7edbbe27825c681a not found: ID does not exist"
Jul 29 19:17:13.703848 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.703779    2579 scope.go:117] "RemoveContainer" containerID="5fff070dc75f865e370f3b1f549124cd9ec17b9714cdc109a7ff703f8cb007b7"
Jul 29 19:17:13.704027 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:17:13.704006    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fff070dc75f865e370f3b1f549124cd9ec17b9714cdc109a7ff703f8cb007b7\": container with ID starting with 5fff070dc75f865e370f3b1f549124cd9ec17b9714cdc109a7ff703f8cb007b7 not found: ID does not exist" containerID="5fff070dc75f865e370f3b1f549124cd9ec17b9714cdc109a7ff703f8cb007b7"
Jul 29 19:17:13.704066 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.704038    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fff070dc75f865e370f3b1f549124cd9ec17b9714cdc109a7ff703f8cb007b7"} err="failed to get container status \"5fff070dc75f865e370f3b1f549124cd9ec17b9714cdc109a7ff703f8cb007b7\": rpc error: code = NotFound desc = could not find container \"5fff070dc75f865e370f3b1f549124cd9ec17b9714cdc109a7ff703f8cb007b7\": container with ID starting with 5fff070dc75f865e370f3b1f549124cd9ec17b9714cdc109a7ff703f8cb007b7 not found: ID does not exist"
Jul 29 19:17:13.704066 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.704058    2579 scope.go:117] "RemoveContainer" containerID="187174b6e557bbbfbe3d4ded5c84a8754dabdd76533d5fc1a4678bcfabb87827"
Jul 29 19:17:13.704326 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:17:13.704310    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"187174b6e557bbbfbe3d4ded5c84a8754dabdd76533d5fc1a4678bcfabb87827\": container with ID starting with 187174b6e557bbbfbe3d4ded5c84a8754dabdd76533d5fc1a4678bcfabb87827 not found: ID does not exist" containerID="187174b6e557bbbfbe3d4ded5c84a8754dabdd76533d5fc1a4678bcfabb87827"
Jul 29 19:17:13.704371 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:13.704330    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"187174b6e557bbbfbe3d4ded5c84a8754dabdd76533d5fc1a4678bcfabb87827"} err="failed to get container status \"187174b6e557bbbfbe3d4ded5c84a8754dabdd76533d5fc1a4678bcfabb87827\": rpc error: code = NotFound desc = could not find container \"187174b6e557bbbfbe3d4ded5c84a8754dabdd76533d5fc1a4678bcfabb87827\": container with ID starting with 187174b6e557bbbfbe3d4ded5c84a8754dabdd76533d5fc1a4678bcfabb87827 not found: ID does not exist"
Jul 29 19:17:14.585850 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:14.585814    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="673008f0-aae4-4afb-8840-717cf3572a1f" path="/var/lib/kubelet/pods/673008f0-aae4-4afb-8840-717cf3572a1f/volumes"
Jul 29 19:17:35.619624 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:35.619588    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"]
Jul 29 19:17:35.620088 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:35.619836    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk" podUID="f29a06ed-7c34-42a3-bce7-216ba08887cb" containerName="main" containerID="cri-o://43d74b9fcab46b4d8645e0eddc4eca3fa1cc0d44abc230a7cea95cf8783eb424" gracePeriod=30
Jul 29 19:17:35.620088 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:35.619940    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk" podUID="f29a06ed-7c34-42a3-bce7-216ba08887cb" containerName="tokenizer" containerID="cri-o://23e744029bfbf7638bac76941370a1309ad935d535be76b3ff7b3987ccfc2b50" gracePeriod=30
Jul 29 19:17:35.775625 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:35.775589    2579 generic.go:356] "Generic (PLEG): container finished" podID="f29a06ed-7c34-42a3-bce7-216ba08887cb" containerID="43d74b9fcab46b4d8645e0eddc4eca3fa1cc0d44abc230a7cea95cf8783eb424" exitCode=0
Jul 29 19:17:35.775792 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:35.775641    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk" event={"ID":"f29a06ed-7c34-42a3-bce7-216ba08887cb","Type":"ContainerDied","Data":"43d74b9fcab46b4d8645e0eddc4eca3fa1cc0d44abc230a7cea95cf8783eb424"}
Jul 29 19:17:36.987983 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:36.987952    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:17:37.011664 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.011628    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f29a06ed-7c34-42a3-bce7-216ba08887cb-tls-certs\") pod \"f29a06ed-7c34-42a3-bce7-216ba08887cb\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") "
Jul 29 19:17:37.011664 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.011668    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-uds\") pod \"f29a06ed-7c34-42a3-bce7-216ba08887cb\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") "
Jul 29 19:17:37.011884 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.011698    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-cache\") pod \"f29a06ed-7c34-42a3-bce7-216ba08887cb\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") "
Jul 29 19:17:37.011884 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.011750    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-tmp\") pod \"f29a06ed-7c34-42a3-bce7-216ba08887cb\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") "
Jul 29 19:17:37.011884 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.011791    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-kserve-provision-location\") pod \"f29a06ed-7c34-42a3-bce7-216ba08887cb\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") "
Jul 29 19:17:37.011884 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.011836    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlvv5\" (UniqueName: \"kubernetes.io/projected/f29a06ed-7c34-42a3-bce7-216ba08887cb-kube-api-access-rlvv5\") pod \"f29a06ed-7c34-42a3-bce7-216ba08887cb\" (UID: \"f29a06ed-7c34-42a3-bce7-216ba08887cb\") "
Jul 29 19:17:37.012041 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.011981    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "f29a06ed-7c34-42a3-bce7-216ba08887cb" (UID: "f29a06ed-7c34-42a3-bce7-216ba08887cb"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:17:37.012095 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.011990    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "f29a06ed-7c34-42a3-bce7-216ba08887cb" (UID: "f29a06ed-7c34-42a3-bce7-216ba08887cb"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:17:37.012157 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.012116    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-uds\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:17:37.012157 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.012116    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "f29a06ed-7c34-42a3-bce7-216ba08887cb" (UID: "f29a06ed-7c34-42a3-bce7-216ba08887cb"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:17:37.012575 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.012552    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "f29a06ed-7c34-42a3-bce7-216ba08887cb" (UID: "f29a06ed-7c34-42a3-bce7-216ba08887cb"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:17:37.014674 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.014482    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f29a06ed-7c34-42a3-bce7-216ba08887cb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f29a06ed-7c34-42a3-bce7-216ba08887cb" (UID: "f29a06ed-7c34-42a3-bce7-216ba08887cb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:17:37.014936 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.014912    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f29a06ed-7c34-42a3-bce7-216ba08887cb-kube-api-access-rlvv5" (OuterVolumeSpecName: "kube-api-access-rlvv5") pod "f29a06ed-7c34-42a3-bce7-216ba08887cb" (UID: "f29a06ed-7c34-42a3-bce7-216ba08887cb"). InnerVolumeSpecName "kube-api-access-rlvv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:17:37.113013 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.112915    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:17:37.113013 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.112953    2579 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-tokenizer-tmp\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:17:37.113013 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.112968    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f29a06ed-7c34-42a3-bce7-216ba08887cb-kserve-provision-location\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:17:37.113013 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.112978    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlvv5\" (UniqueName: \"kubernetes.io/projected/f29a06ed-7c34-42a3-bce7-216ba08887cb-kube-api-access-rlvv5\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:17:37.113013 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.112988    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f29a06ed-7c34-42a3-bce7-216ba08887cb-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:17:37.135944 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.135911    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j85xz/must-gather-hbdld"]
Jul 29 19:17:37.136316 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.136303    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="673008f0-aae4-4afb-8840-717cf3572a1f" containerName="tokenizer"
Jul 29 19:17:37.136355 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.136317    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="673008f0-aae4-4afb-8840-717cf3572a1f" containerName="tokenizer"
Jul 29 19:17:37.136355 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.136338    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="673008f0-aae4-4afb-8840-717cf3572a1f" containerName="main"
Jul 29 19:17:37.136355 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.136344    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="673008f0-aae4-4afb-8840-717cf3572a1f" containerName="main"
Jul 29 19:17:37.136355 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.136354    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="673008f0-aae4-4afb-8840-717cf3572a1f" containerName="storage-initializer"
Jul 29 19:17:37.136482 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.136361    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="673008f0-aae4-4afb-8840-717cf3572a1f" containerName="storage-initializer"
Jul 29 19:17:37.136482 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.136366    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f29a06ed-7c34-42a3-bce7-216ba08887cb" containerName="tokenizer"
Jul 29 19:17:37.136482 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.136373    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f29a06ed-7c34-42a3-bce7-216ba08887cb" containerName="tokenizer"
Jul 29 19:17:37.136482 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.136382    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f29a06ed-7c34-42a3-bce7-216ba08887cb" containerName="storage-initializer"
Jul 29 19:17:37.136482 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.136388    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f29a06ed-7c34-42a3-bce7-216ba08887cb" containerName="storage-initializer"
Jul 29 19:17:37.136482 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.136409    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f29a06ed-7c34-42a3-bce7-216ba08887cb" containerName="main"
Jul 29 19:17:37.136482 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.136416    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f29a06ed-7c34-42a3-bce7-216ba08887cb" containerName="main"
Jul 29 19:17:37.136482 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.136475    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f29a06ed-7c34-42a3-bce7-216ba08887cb" containerName="main"
Jul 29 19:17:37.136482 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.136482    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="673008f0-aae4-4afb-8840-717cf3572a1f" containerName="main"
Jul 29 19:17:37.136773 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.136489    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f29a06ed-7c34-42a3-bce7-216ba08887cb" containerName="tokenizer"
Jul 29 19:17:37.136773 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.136497    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="673008f0-aae4-4afb-8840-717cf3572a1f" containerName="tokenizer"
Jul 29 19:17:37.140116 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.140095    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j85xz/must-gather-hbdld"
Jul 29 19:17:37.144978 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.144956    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-j85xz\"/\"default-dockercfg-rs7rf\""
Jul 29 19:17:37.145101 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.145078    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j85xz\"/\"kube-root-ca.crt\""
Jul 29 19:17:37.145382 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.145367    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j85xz\"/\"openshift-service-ca.crt\""
Jul 29 19:17:37.153383 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.153355    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j85xz/must-gather-hbdld"]
Jul 29 19:17:37.213795 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.213748    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f202afbb-e9cb-4db9-9a5e-d65928dd0106-must-gather-output\") pod \"must-gather-hbdld\" (UID: \"f202afbb-e9cb-4db9-9a5e-d65928dd0106\") " pod="openshift-must-gather-j85xz/must-gather-hbdld"
Jul 29 19:17:37.213958 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.213814    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ktmh\" (UniqueName: \"kubernetes.io/projected/f202afbb-e9cb-4db9-9a5e-d65928dd0106-kube-api-access-4ktmh\") pod \"must-gather-hbdld\" (UID: \"f202afbb-e9cb-4db9-9a5e-d65928dd0106\") " pod="openshift-must-gather-j85xz/must-gather-hbdld"
Jul 29 19:17:37.314614 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.314566    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f202afbb-e9cb-4db9-9a5e-d65928dd0106-must-gather-output\") pod \"must-gather-hbdld\" (UID: \"f202afbb-e9cb-4db9-9a5e-d65928dd0106\") " pod="openshift-must-gather-j85xz/must-gather-hbdld"
Jul 29 19:17:37.314614 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.314616    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ktmh\" (UniqueName: \"kubernetes.io/projected/f202afbb-e9cb-4db9-9a5e-d65928dd0106-kube-api-access-4ktmh\") pod \"must-gather-hbdld\" (UID: \"f202afbb-e9cb-4db9-9a5e-d65928dd0106\") " pod="openshift-must-gather-j85xz/must-gather-hbdld"
Jul 29 19:17:37.314944 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.314922    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f202afbb-e9cb-4db9-9a5e-d65928dd0106-must-gather-output\") pod \"must-gather-hbdld\" (UID: \"f202afbb-e9cb-4db9-9a5e-d65928dd0106\") " pod="openshift-must-gather-j85xz/must-gather-hbdld"
Jul 29 19:17:37.332795 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.332755    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ktmh\" (UniqueName: \"kubernetes.io/projected/f202afbb-e9cb-4db9-9a5e-d65928dd0106-kube-api-access-4ktmh\") pod \"must-gather-hbdld\" (UID: \"f202afbb-e9cb-4db9-9a5e-d65928dd0106\") " pod="openshift-must-gather-j85xz/must-gather-hbdld"
Jul 29 19:17:37.450711 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.450613    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j85xz/must-gather-hbdld"
Jul 29 19:17:37.591610 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.591582    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j85xz/must-gather-hbdld"]
Jul 29 19:17:37.593568 ip-10-0-137-187 kubenswrapper[2579]: W0729 19:17:37.593547    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf202afbb_e9cb_4db9_9a5e_d65928dd0106.slice/crio-458460a0735722fa9ba81bae40d49f6b364034b5e371ea95d403d15e3463dc23 WatchSource:0}: Error finding container 458460a0735722fa9ba81bae40d49f6b364034b5e371ea95d403d15e3463dc23: Status 404 returned error can't find the container with id 458460a0735722fa9ba81bae40d49f6b364034b5e371ea95d403d15e3463dc23
Jul 29 19:17:37.595184 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.595169    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 19:17:37.787114 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.787075    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j85xz/must-gather-hbdld" event={"ID":"f202afbb-e9cb-4db9-9a5e-d65928dd0106","Type":"ContainerStarted","Data":"458460a0735722fa9ba81bae40d49f6b364034b5e371ea95d403d15e3463dc23"}
Jul 29 19:17:37.791706 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.791668    2579 generic.go:356] "Generic (PLEG): container finished" podID="f29a06ed-7c34-42a3-bce7-216ba08887cb" containerID="23e744029bfbf7638bac76941370a1309ad935d535be76b3ff7b3987ccfc2b50" exitCode=0
Jul 29 19:17:37.791892 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.791735    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"
Jul 29 19:17:37.791892 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.791731    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk" event={"ID":"f29a06ed-7c34-42a3-bce7-216ba08887cb","Type":"ContainerDied","Data":"23e744029bfbf7638bac76941370a1309ad935d535be76b3ff7b3987ccfc2b50"}
Jul 29 19:17:37.791892 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.791875    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk" event={"ID":"f29a06ed-7c34-42a3-bce7-216ba08887cb","Type":"ContainerDied","Data":"6072d1e57dbb5a11cafdc0eb569cc8b6ce0aff027a96dfa61386e4ebab110db6"}
Jul 29 19:17:37.792054 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.791910    2579 scope.go:117] "RemoveContainer" containerID="23e744029bfbf7638bac76941370a1309ad935d535be76b3ff7b3987ccfc2b50"
Jul 29 19:17:37.801340 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.801320    2579 scope.go:117] "RemoveContainer" containerID="43d74b9fcab46b4d8645e0eddc4eca3fa1cc0d44abc230a7cea95cf8783eb424"
Jul 29 19:17:37.815877 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.815846    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"]
Jul 29 19:17:37.818478 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.818453    2579 scope.go:117] "RemoveContainer" containerID="7d0f2f72fbe95253eafed78ac48a99442e77fc4d9300d69e7aeab2509b1383f5"
Jul 29 19:17:37.823348 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.823313    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schercptk"]
Jul 29 19:17:37.832411 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.832387    2579 scope.go:117] "RemoveContainer" containerID="23e744029bfbf7638bac76941370a1309ad935d535be76b3ff7b3987ccfc2b50"
Jul 29 19:17:37.832738 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:17:37.832718    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23e744029bfbf7638bac76941370a1309ad935d535be76b3ff7b3987ccfc2b50\": container with ID starting with 23e744029bfbf7638bac76941370a1309ad935d535be76b3ff7b3987ccfc2b50 not found: ID does not exist" containerID="23e744029bfbf7638bac76941370a1309ad935d535be76b3ff7b3987ccfc2b50"
Jul 29 19:17:37.832808 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.832746    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23e744029bfbf7638bac76941370a1309ad935d535be76b3ff7b3987ccfc2b50"} err="failed to get container status \"23e744029bfbf7638bac76941370a1309ad935d535be76b3ff7b3987ccfc2b50\": rpc error: code = NotFound desc = could not find container \"23e744029bfbf7638bac76941370a1309ad935d535be76b3ff7b3987ccfc2b50\": container with ID starting with 23e744029bfbf7638bac76941370a1309ad935d535be76b3ff7b3987ccfc2b50 not found: ID does not exist"
Jul 29 19:17:37.832808 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.832766    2579 scope.go:117] "RemoveContainer" containerID="43d74b9fcab46b4d8645e0eddc4eca3fa1cc0d44abc230a7cea95cf8783eb424"
Jul 29 19:17:37.833013 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:17:37.832998    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43d74b9fcab46b4d8645e0eddc4eca3fa1cc0d44abc230a7cea95cf8783eb424\": container with ID starting with 43d74b9fcab46b4d8645e0eddc4eca3fa1cc0d44abc230a7cea95cf8783eb424 not found: ID does not exist" containerID="43d74b9fcab46b4d8645e0eddc4eca3fa1cc0d44abc230a7cea95cf8783eb424"
Jul 29 19:17:37.833061 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.833016    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43d74b9fcab46b4d8645e0eddc4eca3fa1cc0d44abc230a7cea95cf8783eb424"} err="failed to get container status \"43d74b9fcab46b4d8645e0eddc4eca3fa1cc0d44abc230a7cea95cf8783eb424\": rpc error: code = NotFound desc = could not find container \"43d74b9fcab46b4d8645e0eddc4eca3fa1cc0d44abc230a7cea95cf8783eb424\": container with ID starting with 43d74b9fcab46b4d8645e0eddc4eca3fa1cc0d44abc230a7cea95cf8783eb424 not found: ID does not exist"
Jul 29 19:17:37.833061 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.833029    2579 scope.go:117] "RemoveContainer" containerID="7d0f2f72fbe95253eafed78ac48a99442e77fc4d9300d69e7aeab2509b1383f5"
Jul 29 19:17:37.833215 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:17:37.833192    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d0f2f72fbe95253eafed78ac48a99442e77fc4d9300d69e7aeab2509b1383f5\": container with ID starting with 7d0f2f72fbe95253eafed78ac48a99442e77fc4d9300d69e7aeab2509b1383f5 not found: ID does not exist" containerID="7d0f2f72fbe95253eafed78ac48a99442e77fc4d9300d69e7aeab2509b1383f5"
Jul 29 19:17:37.833258 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:37.833221    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d0f2f72fbe95253eafed78ac48a99442e77fc4d9300d69e7aeab2509b1383f5"} err="failed to get container status \"7d0f2f72fbe95253eafed78ac48a99442e77fc4d9300d69e7aeab2509b1383f5\": rpc error: code = NotFound desc = could not find container \"7d0f2f72fbe95253eafed78ac48a99442e77fc4d9300d69e7aeab2509b1383f5\": container with ID starting with 7d0f2f72fbe95253eafed78ac48a99442e77fc4d9300d69e7aeab2509b1383f5 not found: ID does not exist"
Jul 29 19:17:38.590065 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:38.589521    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f29a06ed-7c34-42a3-bce7-216ba08887cb" path="/var/lib/kubelet/pods/f29a06ed-7c34-42a3-bce7-216ba08887cb/volumes"
Jul 29 19:17:44.834458 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:44.834412    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j85xz/must-gather-hbdld" event={"ID":"f202afbb-e9cb-4db9-9a5e-d65928dd0106","Type":"ContainerStarted","Data":"4a0d874d211d6358e177b4429293c4856a94fc3eb459e92de8d39caf2afa9966"}
Jul 29 19:17:44.834982 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:44.834469    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j85xz/must-gather-hbdld" event={"ID":"f202afbb-e9cb-4db9-9a5e-d65928dd0106","Type":"ContainerStarted","Data":"632f73442ebc2123be0a1900931a5323a262f35b94add6f0dcf46c42f696004d"}
Jul 29 19:17:44.852334 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:17:44.852247    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-j85xz/must-gather-hbdld" podStartSLOduration=1.465063389 podStartE2EDuration="7.85222507s" totalImagesPullingTime="6.387161681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:44.85090324 +0000 UTC m=+2590.802470328" watchObservedRunningTime="2026-07-29 19:17:44.85222507 +0000 UTC m=+2590.803792148"
Jul 29 19:18:08.567213 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:08.567172    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-766fd78c95-kbbsx_abbf5f2c-044a-4007-9656-54b2b1c454fb/router/0.log"
Jul 29 19:18:09.445803 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:09.445772    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-766fd78c95-kbbsx_abbf5f2c-044a-4007-9656-54b2b1c454fb/router/0.log"
Jul 29 19:18:10.262650 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:10.262613    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-867c7768fd-69w2s_041bfccc-5d61-41f9-9062-891c1d136c09/authorino/0.log"
Jul 29 19:18:10.336860 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:10.336835    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-r27zv_12431451-5505-490c-8fa0-ae5aa5e2c7dc/kuadrant-console-plugin/0.log"
Jul 29 19:18:11.985181 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:11.985144    2579 generic.go:356] "Generic (PLEG): container finished" podID="f202afbb-e9cb-4db9-9a5e-d65928dd0106" containerID="632f73442ebc2123be0a1900931a5323a262f35b94add6f0dcf46c42f696004d" exitCode=0
Jul 29 19:18:11.985761 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:11.985223    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j85xz/must-gather-hbdld" event={"ID":"f202afbb-e9cb-4db9-9a5e-d65928dd0106","Type":"ContainerDied","Data":"632f73442ebc2123be0a1900931a5323a262f35b94add6f0dcf46c42f696004d"}
Jul 29 19:18:11.985761 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:11.985652    2579 scope.go:117] "RemoveContainer" containerID="632f73442ebc2123be0a1900931a5323a262f35b94add6f0dcf46c42f696004d"
Jul 29 19:18:12.107679 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:12.107652    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j85xz_must-gather-hbdld_f202afbb-e9cb-4db9-9a5e-d65928dd0106/gather/0.log"
Jul 29 19:18:16.003289 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:16.003243    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-d9d6c_32e483de-2fdb-429d-bb01-5f7ea6a0e46b/global-pull-secret-syncer/0.log"
Jul 29 19:18:16.146618 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:16.146586    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9f7kk_89b15060-8d5e-4bc9-84ba-4a7a4b7cb9d8/konnectivity-agent/0.log"
Jul 29 19:18:16.330868 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:16.330839    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-187.ec2.internal_edd1a1a78d8e70eee8b9b046cdfe44df/haproxy/0.log"
Jul 29 19:18:17.598305 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:17.598270    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-j85xz/must-gather-hbdld"]
Jul 29 19:18:17.598758 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:17.598451    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-j85xz/must-gather-hbdld" podUID="f202afbb-e9cb-4db9-9a5e-d65928dd0106" containerName="copy" containerID="cri-o://4a0d874d211d6358e177b4429293c4856a94fc3eb459e92de8d39caf2afa9966" gracePeriod=2
Jul 29 19:18:17.607209 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:17.607179    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-j85xz/must-gather-hbdld"]
Jul 29 19:18:17.608247 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:18:17.608210    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f202afbb-e9cb-4db9-9a5e-d65928dd0106\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"openshift-must-gather-j85xz\"/\"must-gather-hbdld\": pods \"must-gather-hbdld\" not found" pod="openshift-must-gather-j85xz/must-gather-hbdld"
Jul 29 19:18:17.840691 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:17.840660    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j85xz_must-gather-hbdld_f202afbb-e9cb-4db9-9a5e-d65928dd0106/copy/0.log"
Jul 29 19:18:17.842867 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:17.842846    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j85xz/must-gather-hbdld"
Jul 29 19:18:17.845172 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:18:17.845148    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-hbdld\" is forbidden: User \"system:node:ip-10-0-137-187.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-j85xz\": no relationship found between node 'ip-10-0-137-187.ec2.internal' and this object" podUID="f202afbb-e9cb-4db9-9a5e-d65928dd0106" pod="openshift-must-gather-j85xz/must-gather-hbdld"
Jul 29 19:18:17.895848 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:17.895757    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f202afbb-e9cb-4db9-9a5e-d65928dd0106-must-gather-output\") pod \"f202afbb-e9cb-4db9-9a5e-d65928dd0106\" (UID: \"f202afbb-e9cb-4db9-9a5e-d65928dd0106\") "
Jul 29 19:18:17.895848 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:17.895802    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ktmh\" (UniqueName: \"kubernetes.io/projected/f202afbb-e9cb-4db9-9a5e-d65928dd0106-kube-api-access-4ktmh\") pod \"f202afbb-e9cb-4db9-9a5e-d65928dd0106\" (UID: \"f202afbb-e9cb-4db9-9a5e-d65928dd0106\") "
Jul 29 19:18:17.898120 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:17.898080    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f202afbb-e9cb-4db9-9a5e-d65928dd0106-kube-api-access-4ktmh" (OuterVolumeSpecName: "kube-api-access-4ktmh") pod "f202afbb-e9cb-4db9-9a5e-d65928dd0106" (UID: "f202afbb-e9cb-4db9-9a5e-d65928dd0106"). InnerVolumeSpecName "kube-api-access-4ktmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:18:17.901286 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:17.901259    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f202afbb-e9cb-4db9-9a5e-d65928dd0106-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "f202afbb-e9cb-4db9-9a5e-d65928dd0106" (UID: "f202afbb-e9cb-4db9-9a5e-d65928dd0106"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:18:17.996525 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:17.996467    2579 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f202afbb-e9cb-4db9-9a5e-d65928dd0106-must-gather-output\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:18:17.996525 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:17.996500    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ktmh\" (UniqueName: \"kubernetes.io/projected/f202afbb-e9cb-4db9-9a5e-d65928dd0106-kube-api-access-4ktmh\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\""
Jul 29 19:18:18.017472 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:18.017440    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j85xz_must-gather-hbdld_f202afbb-e9cb-4db9-9a5e-d65928dd0106/copy/0.log"
Jul 29 19:18:18.017854 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:18.017831    2579 generic.go:356] "Generic (PLEG): container finished" podID="f202afbb-e9cb-4db9-9a5e-d65928dd0106" containerID="4a0d874d211d6358e177b4429293c4856a94fc3eb459e92de8d39caf2afa9966" exitCode=143
Jul 29 19:18:18.017913 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:18.017875    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j85xz/must-gather-hbdld"
Jul 29 19:18:18.018693 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:18.017923    2579 scope.go:117] "RemoveContainer" containerID="4a0d874d211d6358e177b4429293c4856a94fc3eb459e92de8d39caf2afa9966"
Jul 29 19:18:18.020752 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:18:18.020729    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-hbdld\" is forbidden: User \"system:node:ip-10-0-137-187.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-j85xz\": no relationship found between node 'ip-10-0-137-187.ec2.internal' and this object" podUID="f202afbb-e9cb-4db9-9a5e-d65928dd0106" pod="openshift-must-gather-j85xz/must-gather-hbdld"
Jul 29 19:18:18.029777 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:18.027795    2579 scope.go:117] "RemoveContainer" containerID="632f73442ebc2123be0a1900931a5323a262f35b94add6f0dcf46c42f696004d"
Jul 29 19:18:18.029777 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:18:18.028801    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-hbdld\" is forbidden: User \"system:node:ip-10-0-137-187.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-j85xz\": no relationship found between node 'ip-10-0-137-187.ec2.internal' and this object" podUID="f202afbb-e9cb-4db9-9a5e-d65928dd0106" pod="openshift-must-gather-j85xz/must-gather-hbdld"
Jul 29 19:18:18.042626 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:18.042600    2579 scope.go:117] "RemoveContainer" containerID="4a0d874d211d6358e177b4429293c4856a94fc3eb459e92de8d39caf2afa9966"
Jul 29 19:18:18.042957 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:18:18.042937    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a0d874d211d6358e177b4429293c4856a94fc3eb459e92de8d39caf2afa9966\": container with ID starting with 4a0d874d211d6358e177b4429293c4856a94fc3eb459e92de8d39caf2afa9966 not found: ID does not exist" containerID="4a0d874d211d6358e177b4429293c4856a94fc3eb459e92de8d39caf2afa9966"
Jul 29 19:18:18.043044 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:18.042971    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a0d874d211d6358e177b4429293c4856a94fc3eb459e92de8d39caf2afa9966"} err="failed to get container status \"4a0d874d211d6358e177b4429293c4856a94fc3eb459e92de8d39caf2afa9966\": rpc error: code = NotFound desc = could not find container \"4a0d874d211d6358e177b4429293c4856a94fc3eb459e92de8d39caf2afa9966\": container with ID starting with 4a0d874d211d6358e177b4429293c4856a94fc3eb459e92de8d39caf2afa9966 not found: ID does not exist"
Jul 29 19:18:18.043044 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:18.042997    2579 scope.go:117] "RemoveContainer" containerID="632f73442ebc2123be0a1900931a5323a262f35b94add6f0dcf46c42f696004d"
Jul 29 19:18:18.043277 ip-10-0-137-187 kubenswrapper[2579]: E0729 19:18:18.043255    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"632f73442ebc2123be0a1900931a5323a262f35b94add6f0dcf46c42f696004d\": container with ID starting with 632f73442ebc2123be0a1900931a5323a262f35b94add6f0dcf46c42f696004d not found: ID does not exist" containerID="632f73442ebc2123be0a1900931a5323a262f35b94add6f0dcf46c42f696004d"
Jul 29 19:18:18.043316 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:18.043283    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"632f73442ebc2123be0a1900931a5323a262f35b94add6f0dcf46c42f696004d"} err="failed to get container status \"632f73442ebc2123be0a1900931a5323a262f35b94add6f0dcf46c42f696004d\": rpc error: code = NotFound desc = could not find container \"632f73442ebc2123be0a1900931a5323a262f35b94add6f0dcf46c42f696004d\": container with ID starting with 632f73442ebc2123be0a1900931a5323a262f35b94add6f0dcf46c42f696004d not found: ID does not exist"
Jul 29 19:18:18.586368 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:18.586326    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f202afbb-e9cb-4db9-9a5e-d65928dd0106" path="/var/lib/kubelet/pods/f202afbb-e9cb-4db9-9a5e-d65928dd0106/volumes"
Jul 29 19:18:20.330940 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:20.330901    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-867c7768fd-69w2s_041bfccc-5d61-41f9-9062-891c1d136c09/authorino/0.log"
Jul 29 19:18:20.436142 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:20.436111    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-r27zv_12431451-5505-490c-8fa0-ae5aa5e2c7dc/kuadrant-console-plugin/0.log"
Jul 29 19:18:22.122469 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:22.122431    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8a7041c-2735-41fa-a484-0c3bf995c310/alertmanager/0.log"
Jul 29 19:18:22.154491 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:22.154463    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8a7041c-2735-41fa-a484-0c3bf995c310/config-reloader/0.log"
Jul 29 19:18:22.182241 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:22.182209    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8a7041c-2735-41fa-a484-0c3bf995c310/kube-rbac-proxy-web/0.log"
Jul 29 19:18:22.214639 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:22.214604    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8a7041c-2735-41fa-a484-0c3bf995c310/kube-rbac-proxy/0.log"
Jul 29 19:18:22.247480 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:22.247450    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8a7041c-2735-41fa-a484-0c3bf995c310/kube-rbac-proxy-metric/0.log"
Jul 29 19:18:22.275230 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:22.275199    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8a7041c-2735-41fa-a484-0c3bf995c310/prom-label-proxy/0.log"
Jul 29 19:18:22.302770 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:22.302738    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8a7041c-2735-41fa-a484-0c3bf995c310/init-config-reloader/0.log"
Jul 29 19:18:22.371785 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:22.371751    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gzvjm_aedf1ecb-e610-4bf3-9688-a969c14dd65b/cluster-monitoring-operator/0.log"
Jul 29 19:18:22.494607 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:22.494495    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-86969d99b9-xcn44_4ea7a10f-8208-4922-998e-0e38f321dbce/metrics-server/0.log"
Jul 29 19:18:22.529580 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:22.529546    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-65vb9_b2bdaf43-1546-44b4-834b-810af7489214/monitoring-plugin/0.log"
Jul 29 19:18:22.799712 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:22.799673    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fq9lf_53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d/node-exporter/0.log"
Jul 29 19:18:22.822261 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:22.822234    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fq9lf_53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d/kube-rbac-proxy/0.log"
Jul 29 19:18:22.849444 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:22.849419    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fq9lf_53d0338a-dbb0-4beb-8ec5-8ef3fd46b63d/init-textfile/0.log"
Jul 29 19:18:22.883001 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:22.882972    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-6p9hg_5e4c363b-bd03-40d1-b561-7be506768d71/kube-rbac-proxy-main/0.log"
Jul 29 19:18:22.912754 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:22.912722    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-6p9hg_5e4c363b-bd03-40d1-b561-7be506768d71/kube-rbac-proxy-self/0.log"
Jul 29 19:18:22.938706 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:22.938680    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-6p9hg_5e4c363b-bd03-40d1-b561-7be506768d71/openshift-state-metrics/0.log"
Jul 29 19:18:23.233705 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:23.233679    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-ndnmt_1de2e57e-5f4f-4db4-9be9-bfb561e48e82/prometheus-operator/0.log"
Jul 29 19:18:23.257096 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:23.257068    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-ndnmt_1de2e57e-5f4f-4db4-9be9-bfb561e48e82/kube-rbac-proxy/0.log"
Jul 29 19:18:23.288776 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:23.288746    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-hf46b_1bd334fd-5fba-4ee9-a95f-ed7957bec060/prometheus-operator-admission-webhook/0.log"
Jul 29 19:18:23.334419 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:23.334330    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6769746546-9s2gd_da9d595b-54f2-49b2-8c36-03937644edc4/telemeter-client/0.log"
Jul 29 19:18:23.358136 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:23.358099    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6769746546-9s2gd_da9d595b-54f2-49b2-8c36-03937644edc4/reload/0.log"
Jul 29 19:18:23.387863 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:23.387836    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6769746546-9s2gd_da9d595b-54f2-49b2-8c36-03937644edc4/kube-rbac-proxy/0.log"
Jul 29 19:18:23.418466 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:23.418435    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56dbd69f55-xnzw2_0cf3e6f3-df70-4246-bdae-e2c16ba98c34/thanos-query/0.log"
Jul 29 19:18:23.441581 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:23.441536    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56dbd69f55-xnzw2_0cf3e6f3-df70-4246-bdae-e2c16ba98c34/kube-rbac-proxy-web/0.log"
Jul 29 19:18:23.467635 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:23.467608    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56dbd69f55-xnzw2_0cf3e6f3-df70-4246-bdae-e2c16ba98c34/kube-rbac-proxy/0.log"
Jul 29 19:18:23.493488 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:23.493456    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56dbd69f55-xnzw2_0cf3e6f3-df70-4246-bdae-e2c16ba98c34/prom-label-proxy/0.log"
Jul 29 19:18:23.516966 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:23.516937    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56dbd69f55-xnzw2_0cf3e6f3-df70-4246-bdae-e2c16ba98c34/kube-rbac-proxy-rules/0.log"
Jul 29 19:18:23.541413 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:23.541383    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56dbd69f55-xnzw2_0cf3e6f3-df70-4246-bdae-e2c16ba98c34/kube-rbac-proxy-metrics/0.log"
Jul 29 19:18:24.786025 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:24.785995    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-ptp2l_7c6804ba-7543-4df8-8ca8-d5b463078329/networking-console-plugin/0.log"
Jul 29 19:18:25.315400 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:25.315355    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/1.log"
Jul 29 19:18:25.325466 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:25.325440    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-f8wr7_272b03c5-141c-4a65-b707-e1d547ceab87/console-operator/2.log"
Jul 29 19:18:25.811739 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:25.811693    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-878ht_9eeaeb36-0549-4b09-9663-7d77720fd5b4/download-server/0.log"
Jul 29 19:18:26.057217 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.057185    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"]
Jul 29 19:18:26.057791 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.057776    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f202afbb-e9cb-4db9-9a5e-d65928dd0106" containerName="gather"
Jul 29 19:18:26.057835 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.057795    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f202afbb-e9cb-4db9-9a5e-d65928dd0106" containerName="gather"
Jul 29 19:18:26.057835 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.057806    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f202afbb-e9cb-4db9-9a5e-d65928dd0106" containerName="copy"
Jul 29 19:18:26.057835 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.057815    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f202afbb-e9cb-4db9-9a5e-d65928dd0106" containerName="copy"
Jul 29 19:18:26.057933 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.057915    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f202afbb-e9cb-4db9-9a5e-d65928dd0106" containerName="copy"
Jul 29 19:18:26.057933 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.057930    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f202afbb-e9cb-4db9-9a5e-d65928dd0106" containerName="gather"
Jul 29 19:18:26.063565 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.063485    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.066201 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.066181    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xjw8s\"/\"kube-root-ca.crt\""
Jul 29 19:18:26.067581 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.067561    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xjw8s\"/\"openshift-service-ca.crt\""
Jul 29 19:18:26.067675 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.067568    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xjw8s\"/\"default-dockercfg-z2hkg\""
Jul 29 19:18:26.073239 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.073217    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"]
Jul 29 19:18:26.165463 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.165426    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6501de5d-a6d0-42ec-8c16-82f492ea64d7-sys\") pod \"perf-node-gather-daemonset-d6rw7\" (UID: \"6501de5d-a6d0-42ec-8c16-82f492ea64d7\") " pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.165463 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.165463    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6501de5d-a6d0-42ec-8c16-82f492ea64d7-proc\") pod \"perf-node-gather-daemonset-d6rw7\" (UID: \"6501de5d-a6d0-42ec-8c16-82f492ea64d7\") " pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.165742 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.165493    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6501de5d-a6d0-42ec-8c16-82f492ea64d7-lib-modules\") pod \"perf-node-gather-daemonset-d6rw7\" (UID: \"6501de5d-a6d0-42ec-8c16-82f492ea64d7\") " pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.165742 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.165567    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfqmw\" (UniqueName: \"kubernetes.io/projected/6501de5d-a6d0-42ec-8c16-82f492ea64d7-kube-api-access-rfqmw\") pod \"perf-node-gather-daemonset-d6rw7\" (UID: \"6501de5d-a6d0-42ec-8c16-82f492ea64d7\") " pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.165742 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.165586    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6501de5d-a6d0-42ec-8c16-82f492ea64d7-podres\") pod \"perf-node-gather-daemonset-d6rw7\" (UID: \"6501de5d-a6d0-42ec-8c16-82f492ea64d7\") " pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.266793 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.266754    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfqmw\" (UniqueName: \"kubernetes.io/projected/6501de5d-a6d0-42ec-8c16-82f492ea64d7-kube-api-access-rfqmw\") pod \"perf-node-gather-daemonset-d6rw7\" (UID: \"6501de5d-a6d0-42ec-8c16-82f492ea64d7\") " pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.266793 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.266801    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6501de5d-a6d0-42ec-8c16-82f492ea64d7-podres\") pod \"perf-node-gather-daemonset-d6rw7\" (UID: \"6501de5d-a6d0-42ec-8c16-82f492ea64d7\") " pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.267075 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.266917    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6501de5d-a6d0-42ec-8c16-82f492ea64d7-sys\") pod \"perf-node-gather-daemonset-d6rw7\" (UID: \"6501de5d-a6d0-42ec-8c16-82f492ea64d7\") " pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.267075 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.266940    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6501de5d-a6d0-42ec-8c16-82f492ea64d7-proc\") pod \"perf-node-gather-daemonset-d6rw7\" (UID: \"6501de5d-a6d0-42ec-8c16-82f492ea64d7\") " pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.267075 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.266976    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6501de5d-a6d0-42ec-8c16-82f492ea64d7-lib-modules\") pod \"perf-node-gather-daemonset-d6rw7\" (UID: \"6501de5d-a6d0-42ec-8c16-82f492ea64d7\") " pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.267075 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.267023    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6501de5d-a6d0-42ec-8c16-82f492ea64d7-podres\") pod \"perf-node-gather-daemonset-d6rw7\" (UID: \"6501de5d-a6d0-42ec-8c16-82f492ea64d7\") " pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.267075 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.267040    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6501de5d-a6d0-42ec-8c16-82f492ea64d7-sys\") pod \"perf-node-gather-daemonset-d6rw7\" (UID: \"6501de5d-a6d0-42ec-8c16-82f492ea64d7\") " pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.267075 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.267047    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6501de5d-a6d0-42ec-8c16-82f492ea64d7-proc\") pod \"perf-node-gather-daemonset-d6rw7\" (UID: \"6501de5d-a6d0-42ec-8c16-82f492ea64d7\") " pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.267301 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.267120    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6501de5d-a6d0-42ec-8c16-82f492ea64d7-lib-modules\") pod \"perf-node-gather-daemonset-d6rw7\" (UID: \"6501de5d-a6d0-42ec-8c16-82f492ea64d7\") " pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.276381 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.276352    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfqmw\" (UniqueName: \"kubernetes.io/projected/6501de5d-a6d0-42ec-8c16-82f492ea64d7-kube-api-access-rfqmw\") pod \"perf-node-gather-daemonset-d6rw7\" (UID: \"6501de5d-a6d0-42ec-8c16-82f492ea64d7\") " pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.374600 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.374491    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:26.444292 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.444260    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-wwht4_33d38e5d-13d5-45d9-bc61-fb9ba537556b/volume-data-source-validator/0.log"
Jul 29 19:18:26.512735 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:26.512708    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"]
Jul 29 19:18:26.515182 ip-10-0-137-187 kubenswrapper[2579]: W0729 19:18:26.515153    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6501de5d_a6d0_42ec_8c16_82f492ea64d7.slice/crio-266ca76911ee18c19336375123ee73d828e50d22dae0f4e42ed1923785185fee WatchSource:0}: Error finding container 266ca76911ee18c19336375123ee73d828e50d22dae0f4e42ed1923785185fee: Status 404 returned error can't find the container with id 266ca76911ee18c19336375123ee73d828e50d22dae0f4e42ed1923785185fee
Jul 29 19:18:27.064587 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:27.064549    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7" event={"ID":"6501de5d-a6d0-42ec-8c16-82f492ea64d7","Type":"ContainerStarted","Data":"a6d3b42852b381662b4fe0f7247f0ff0e8ba62fff40ec6406c8e1f468529137b"}
Jul 29 19:18:27.065062 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:27.064604    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7" event={"ID":"6501de5d-a6d0-42ec-8c16-82f492ea64d7","Type":"ContainerStarted","Data":"266ca76911ee18c19336375123ee73d828e50d22dae0f4e42ed1923785185fee"}
Jul 29 19:18:27.065062 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:27.064635    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:27.082611 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:27.082560    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7" podStartSLOduration=1.082544764 podStartE2EDuration="1.082544764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:18:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:27.081200108 +0000 UTC m=+2633.032767185" watchObservedRunningTime="2026-07-29 19:18:27.082544764 +0000 UTC m=+2633.034111841"
Jul 29 19:18:27.511266 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:27.511243    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kdnwg_66f3a250-259a-4db3-8a53-bba41080d0cd/dns/0.log"
Jul 29 19:18:27.536667 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:27.536645    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kdnwg_66f3a250-259a-4db3-8a53-bba41080d0cd/kube-rbac-proxy/0.log"
Jul 29 19:18:27.702618 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:27.702587    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bd5l5_d1aa370e-70cd-4cbc-9af4-508b7a39b9fc/dns-node-resolver/0.log"
Jul 29 19:18:28.342119 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:28.342085    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6vrk4_dcfea240-b8d3-4728-b24f-4fe8268f6d05/node-ca/0.log"
Jul 29 19:18:29.448158 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:29.448127    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-766fd78c95-kbbsx_abbf5f2c-044a-4007-9656-54b2b1c454fb/router/0.log"
Jul 29 19:18:29.970876 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:29.970849    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6z9cl_0dd1da74-2c12-4022-be1e-3b0879d4c0ba/serve-healthcheck-canary/0.log"
Jul 29 19:18:30.507196 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:30.507163    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-pzkcg_1e7b645a-55d5-430f-a998-91b89c45b885/insights-operator/0.log"
Jul 29 19:18:30.509739 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:30.509713    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-pzkcg_1e7b645a-55d5-430f-a998-91b89c45b885/insights-operator/1.log"
Jul 29 19:18:30.705662 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:30.705633    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cc6bt_b0cbd483-407c-4f29-a033-fa0beda9bba7/kube-rbac-proxy/0.log"
Jul 29 19:18:30.730414 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:30.730388    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cc6bt_b0cbd483-407c-4f29-a033-fa0beda9bba7/exporter/0.log"
Jul 29 19:18:30.761575 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:30.761545    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cc6bt_b0cbd483-407c-4f29-a033-fa0beda9bba7/extractor/0.log"
Jul 29 19:18:33.079557 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:33.079526    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xjw8s/perf-node-gather-daemonset-d6rw7"
Jul 29 19:18:33.486231 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:33.486109    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7698b7b56c-vqmtr_e1eef13b-9722-45d0-a109-2f07e7b3639a/manager/0.log"
Jul 29 19:18:33.544790 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:33.544761    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-xn9rn_97fe6477-1ad4-4896-a53c-7bd8124c2e6d/openshift-lws-operator/0.log"
Jul 29 19:18:37.119811 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:37.119781    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-766fd78c95-kbbsx_abbf5f2c-044a-4007-9656-54b2b1c454fb/router/0.log"
Jul 29 19:18:38.282823 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:38.282790    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-766fd78c95-kbbsx_abbf5f2c-044a-4007-9656-54b2b1c454fb/router/0.log"
Jul 29 19:18:41.347906 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:41.347873    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2kjtm_98679e95-83f4-48b6-8295-f097eebbd6bb/kube-multus-additional-cni-plugins/0.log"
Jul 29 19:18:41.373805 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:41.373772    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2kjtm_98679e95-83f4-48b6-8295-f097eebbd6bb/egress-router-binary-copy/0.log"
Jul 29 19:18:41.398758 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:41.398726    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2kjtm_98679e95-83f4-48b6-8295-f097eebbd6bb/cni-plugins/0.log"
Jul 29 19:18:41.423259 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:41.423229    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2kjtm_98679e95-83f4-48b6-8295-f097eebbd6bb/bond-cni-plugin/0.log"
Jul 29 19:18:41.447676 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:41.447635    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2kjtm_98679e95-83f4-48b6-8295-f097eebbd6bb/routeoverride-cni/0.log"
Jul 29 19:18:41.473862 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:41.473833    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2kjtm_98679e95-83f4-48b6-8295-f097eebbd6bb/whereabouts-cni-bincopy/0.log"
Jul 29 19:18:41.500847 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:41.500817    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2kjtm_98679e95-83f4-48b6-8295-f097eebbd6bb/whereabouts-cni/0.log"
Jul 29 19:18:41.914193 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:41.914146    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nnckl_8277d741-f79d-4ff1-bf93-7b8443daa3be/kube-multus/0.log"
Jul 29 19:18:42.062052 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:42.062029    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2vbxt_ef2dbc84-9591-4b1b-8613-7887422229f2/network-metrics-daemon/0.log"
Jul 29 19:18:42.083095 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:42.083063    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2vbxt_ef2dbc84-9591-4b1b-8613-7887422229f2/kube-rbac-proxy/0.log"
Jul 29 19:18:43.018384 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:43.018347    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-998nt_a7729ae2-a418-4c3a-b3b5-1c8dfec9347e/ovn-controller/0.log"
Jul 29 19:18:43.040722 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:43.040694    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-998nt_a7729ae2-a418-4c3a-b3b5-1c8dfec9347e/ovn-acl-logging/0.log"
Jul 29 19:18:43.069419 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:43.069390    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-998nt_a7729ae2-a418-4c3a-b3b5-1c8dfec9347e/kube-rbac-proxy-node/0.log"
Jul 29 19:18:43.095132 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:43.095101    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-998nt_a7729ae2-a418-4c3a-b3b5-1c8dfec9347e/kube-rbac-proxy-ovn-metrics/0.log"
Jul 29 19:18:43.115815 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:43.115787    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-998nt_a7729ae2-a418-4c3a-b3b5-1c8dfec9347e/northd/0.log"
Jul 29 19:18:43.140295 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:43.140265    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-998nt_a7729ae2-a418-4c3a-b3b5-1c8dfec9347e/nbdb/0.log"
Jul 29 19:18:43.168132 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:43.168099    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-998nt_a7729ae2-a418-4c3a-b3b5-1c8dfec9347e/sbdb/0.log"
Jul 29 19:18:43.239763 ip-10-0-137-187 kubenswrapper[2579]: I0729 19:18:43.239728    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-998nt_a7729ae2-a418-4c3a-b3b5-1c8dfec9347e/ovnkube-controller/0.log"