Oct 30 22:07:52.906940 ip-10-0-138-202 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 22:07:53.412442 ip-10-0-138-202 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:07:53.412442 ip-10-0-138-202 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 22:07:53.412442 ip-10-0-138-202 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:07:53.412914 ip-10-0-138-202 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:07:53.415519 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415446    2568 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:07:53.415519 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415519    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415523    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415526    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415529    2568 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415532    2568 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415534    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415537    2568 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415539    2568 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415542    2568 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415544    2568 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415547    2568 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415550    2568 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415552    2568 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415555    2568 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415557    2568 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415561    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415563    2568 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415566    2568 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415568    2568 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:07:53.415582 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415571    2568 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415573    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415575    2568 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415578    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415580    2568 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415583    2568 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415588    2568 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415593    2568 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415596    2568 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415599    2568 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415601    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415604    2568 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415607    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415609    2568 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415612    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415614    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415617    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415620    2568 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415622    2568 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415624    2568 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:07:53.416072 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415627    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415629    2568 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415632    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415634    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415657    2568 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415661    2568 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415664    2568 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415667    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415669    2568 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415672    2568 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415675    2568 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415679    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415681    2568 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415684    2568 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415688    2568 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415691    2568 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415693    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415696    2568 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415698    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:07:53.416805 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415701    2568 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415704    2568 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415707    2568 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415709    2568 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415711    2568 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415714    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415716    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415720    2568 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415723    2568 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415725    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415728    2568 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415730    2568 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415733    2568 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415735    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415738    2568 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415741    2568 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415743    2568 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415746    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415749    2568 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415752    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:07:53.417262 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415755    2568 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415757    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415760    2568 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415763    2568 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415766    2568 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415768    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415771    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415773    2568 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415776    2568 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415779    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415781    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415784    2568 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415786    2568 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415789    2568 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415792    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415794    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415797    2568 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415799    2568 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415802    2568 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:07:53.417772 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415804    2568 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415807    2568 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415810    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415813    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415815    2568 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415817    2568 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415820    2568 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.415823    2568 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.417981    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.417993    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.417997    2568 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418000    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418004    2568 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418007    2568 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418010    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418012    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418015    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418017    2568 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418020    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:07:53.418228 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418023    2568 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418025    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418028    2568 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418030    2568 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418033    2568 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418035    2568 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418038    2568 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418040    2568 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418043    2568 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418045    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418048    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418051    2568 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418053    2568 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418056    2568 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418058    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418061    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418064    2568 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418067    2568 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418072    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:07:53.418721 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418078    2568 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418081    2568 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418085    2568 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418088    2568 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418091    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418094    2568 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418096    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418099    2568 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418101    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418104    2568 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418106    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418108    2568 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418111    2568 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418113    2568 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418116    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418119    2568 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418121    2568 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418124    2568 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418126    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418129    2568 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:07:53.419171 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418131    2568 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418134    2568 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418136    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418139    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418141    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418148    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418151    2568 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418153    2568 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418156    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418159    2568 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418161    2568 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418164    2568 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418166    2568 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418169    2568 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418171    2568 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418174    2568 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418176    2568 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418179    2568 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418182    2568 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418184    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:07:53.419635 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418187    2568 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418189    2568 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418192    2568 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418194    2568 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418197    2568 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418199    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418201    2568 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418204    2568 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418206    2568 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418209    2568 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418212    2568 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418216    2568 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418219    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418222    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418225    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418227    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418230    2568 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418233    2568 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:07:53.420243 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418235    2568 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418239    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418241    2568 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418244    2568 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418246    2568 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418249    2568 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418251    2568 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418254    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418257    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418260    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418263    2568 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418265    2568 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418268    2568 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418270    2568 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418273    2568 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418276    2568 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418279    2568 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418281    2568 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418790    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418796    2568 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:07:53.420695 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418799    2568 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418802    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418805    2568 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418808    2568 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418810    2568 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418813    2568 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418815    2568 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418818    2568 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418821    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418824    2568 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418826    2568 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418831    2568 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418834    2568 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418837    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418841    2568 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418845    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418848    2568 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418851    2568 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:07:53.421162 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418854    2568 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418857    2568 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418859    2568 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418862    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418864    2568 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418867    2568 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418870    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418872    2568 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418874    2568 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418877    2568 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418879    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418882    2568 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418884    2568 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418887    2568 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418890    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418892    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418895    2568 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418897    2568 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418900    2568 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418905    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:07:53.421590 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418907    2568 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418910    2568 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418913    2568 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418915    2568 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418918    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418921    2568 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418924    2568 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418926    2568 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418929    2568 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418931    2568 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418934    2568 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418937    2568 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418939    2568 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418941    2568 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418944    2568 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418946    2568 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418949    2568 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418951    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418954    2568 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418956    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:07:53.422062 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418959    2568 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418961    2568 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418964    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418966    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418968    2568 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418971    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418973    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418976    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418978    2568 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418980    2568 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418983    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418986    2568 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418989    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418992    2568 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418994    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.418997    2568 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419000    2568 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419003    2568 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419005    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419008    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:07:53.422530 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419010    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419013    2568 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419016    2568 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419018    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419020    2568 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419023    2568 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419025    2568 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419028    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419030    2568 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419033    2568 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419035    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419038    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419040    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419042    2568 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419045    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419047    2568 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419049    2568 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419052    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419054    2568 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419057    2568 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:07:53.423024 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419059    2568 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419062    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419064    2568 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419067    2568 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419070    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.419073    2568 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420054    2568 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420065    2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420073    2568 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420078    2568 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420082    2568 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420086    2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420090    2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420094    2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420097    2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420101    2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420104    2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420107    2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420110    2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420113    2568 flags.go:64] FLAG: --cgroup-root=""
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420116    2568 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420119    2568 flags.go:64] FLAG: --client-ca-file=""
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420122    2568 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420124    2568 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 22:07:53.423486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420128    2568 flags.go:64] FLAG: --cluster-domain=""
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420131    2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420134    2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420137    2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420141    2568 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420144    2568 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420148    2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420151    2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420154    2568 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420157    2568 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420160    2568 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420164    2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420167    2568 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420170    2568 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420174    2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420177    2568 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420180    2568 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420185    2568 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420188    2568 flags.go:64] FLAG: --enable-server="true"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420191    2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420196    2568 flags.go:64] FLAG: --event-burst="100"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420199    2568 flags.go:64] FLAG: --event-qps="50"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420202    2568 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420205    2568 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 22:07:53.424048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420208    2568 flags.go:64] FLAG: --eviction-hard=""
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420212    2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420215    2568 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420237    2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420241    2568 flags.go:64] FLAG: --eviction-soft=""
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420245    2568 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420248    2568 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420252    2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420255    2568 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420258    2568 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420261    2568 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420264    2568 flags.go:64] FLAG: --feature-gates=""
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420268    2568 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420271    2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420275    2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420278    2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420281    2568 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420284    2568 flags.go:64] FLAG: --help="false"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420287    2568 flags.go:64] FLAG: --hostname-override="ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420290    2568 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420293    2568 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420296    2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420300    2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 22:07:53.424670 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420303    2568 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420306    2568 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420312    2568 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420315    2568 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420318    2568 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420322    2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420325    2568 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420328    2568 flags.go:64] FLAG: --kube-reserved=""
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420330    2568 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420333    2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420336    2568 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420339    2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420342    2568 flags.go:64] FLAG: --lock-file=""
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420345    2568 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420348    2568 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420351    2568 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420356    2568 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420359    2568 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420362    2568 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420365    2568 flags.go:64] FLAG: --logging-format="text"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420368    2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420371    2568 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420374    2568 flags.go:64] FLAG: --manifest-url=""
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420377    2568 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420382    2568 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 22:07:53.425225 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420385    2568 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420389    2568 flags.go:64] FLAG: --max-pods="110"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420392    2568 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420396    2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420399    2568 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420402    2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420405    2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420408    2568 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420411    2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420419    2568 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420422    2568 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420426    2568 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420429    2568 flags.go:64] FLAG: --pod-cidr=""
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420432    2568 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420435    2568 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420438    2568 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420441    2568 flags.go:64] FLAG: --port="10250"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420444    2568 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420447    2568 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-060bd5c44b3270e14"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420450    2568 flags.go:64] FLAG: --qos-reserved=""
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420453    2568 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420456    2568 flags.go:64] FLAG: --register-node="true"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420458    2568 flags.go:64] FLAG: --register-with-taints=""
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420462    2568 flags.go:64] FLAG: --registry-burst="10"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420465    2568 flags.go:64] FLAG: --registry-qps="5"
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420468    2568 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 22:07:53.425856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420470    2568 flags.go:64] FLAG: --reserved-memory=""
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420474    2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420477    2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420480    2568 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420483    2568 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420485    2568 flags.go:64] FLAG: --runonce="false"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420488    2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420491    2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420494    2568 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420497    2568 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420500    2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420503    2568 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420506    2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420509    2568 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420512    2568 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420516    2568 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420519    2568 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420522    2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420526    2568 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420529    2568 flags.go:64] FLAG: --system-cgroups=""
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420532    2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420537    2568 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420540    2568 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420542    2568 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420546    2568 flags.go:64] FLAG: --tls-min-version=""
Oct 30 22:07:53.426551 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420549    2568 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420552    2568 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420555    2568 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420558    2568 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420561    2568 flags.go:64] FLAG: --v="2"
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420565    2568 flags.go:64] FLAG: --version="false"
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420569    2568 flags.go:64] FLAG: --vmodule=""
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420574    2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.420577    2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420775    2568 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420780    2568 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420784    2568 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420787    2568 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420803    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420807    2568 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420810    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420813    2568 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420816    2568 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420818    2568 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420821    2568 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420824    2568 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:07:53.427152 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420827    2568 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420831    2568 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420833    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420836    2568 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420839    2568 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420842    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420844    2568 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420847    2568 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420850    2568 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420852    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420855    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420857    2568 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420860    2568 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420862    2568 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420865    2568 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420867    2568 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420870    2568 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420872    2568 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420876    2568 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420879    2568 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:07:53.427652 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420881    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420883    2568 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420887    2568 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420891    2568 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420894    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420897    2568 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420899    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420902    2568 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420904    2568 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420907    2568 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420909    2568 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420911    2568 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420914    2568 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420917    2568 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420919    2568 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420922    2568 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420924    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420927    2568 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420930    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420933    2568 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:07:53.428127 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420935    2568 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420937    2568 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420940    2568 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420942    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420945    2568 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420947    2568 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420949    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420952    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420954    2568 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420957    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420961    2568 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420963    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420966    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420968    2568 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420971    2568 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420973    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420975    2568 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420978    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420980    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:07:53.428597 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420983    2568 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420985    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420988    2568 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420990    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420993    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420995    2568 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.420998    2568 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421000    2568 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421003    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421006    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421008    2568 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421011    2568 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421014    2568 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421016    2568 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421019    2568 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421021    2568 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421024    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421026    2568 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421028    2568 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421031    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:07:53.429103 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421033    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:07:53.429604 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421036    2568 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:07:53.429604 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421038    2568 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:07:53.429604 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421041    2568 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:07:53.429604 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421044    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:07:53.429604 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421046    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:07:53.429604 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421049    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:07:53.429604 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421051    2568 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:07:53.429604 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421054    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:07:53.429604 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421057    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:07:53.429604 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421059    2568 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:07:53.429604 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421063    2568 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:07:53.429604 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421066    2568 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:07:53.429604 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421070    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:07:53.429604 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421072    2568 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:07:53.429604 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.421100    2568 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421138    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421142    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421145    2568 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421148    2568 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421151    2568 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421154    2568 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421170    2568 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421174    2568 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421177    2568 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421181    2568 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421184    2568 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421186    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421189    2568 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421193    2568 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421197    2568 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421200    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421202    2568 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421205    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:07:53.430048 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421207    2568 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421211    2568 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421213    2568 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421216    2568 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421218    2568 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421221    2568 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421223    2568 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421226    2568 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421228    2568 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421231    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421234    2568 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421236    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421238    2568 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421241    2568 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421243    2568 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421246    2568 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421249    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421252    2568 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421254    2568 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421257    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:07:53.430491 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421259    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421267    2568 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421269    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421272    2568 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421275    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421277    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421280    2568 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421282    2568 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421285    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421287    2568 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421290    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421292    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421294    2568 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421298    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421301    2568 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421303    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421306    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421308    2568 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421310    2568 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:07:53.430972 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421313    2568 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421325    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421327    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421330    2568 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421332    2568 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421335    2568 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421337    2568 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421340    2568 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421342    2568 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421346    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421348    2568 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421351    2568 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421353    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421356    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421358    2568 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421361    2568 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421363    2568 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421366    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421369    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421372    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:07:53.431462 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421374    2568 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421377    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421380    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421383    2568 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421386    2568 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421390    2568 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421394    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421397    2568 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421399    2568 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421402    2568 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421405    2568 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421407    2568 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421410    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421412    2568 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421415    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421417    2568 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421420    2568 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421422    2568 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421425    2568 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:07:53.431948 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421427    2568 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421430    2568 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421433    2568 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421436    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421438    2568 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421441    2568 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421443    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421446    2568 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421449    2568 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.421451    2568 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.428011    2568 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.428027    2568 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.428043    2568 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.428048    2568 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428102    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428108    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428111    2568 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428115    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428118    2568 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:07:53.432407 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428121    2568 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428124    2568 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428127    2568 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428129    2568 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428132    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428135    2568 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428138    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428141    2568 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428143    2568 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428146    2568 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428148    2568 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428151    2568 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428154    2568 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428156    2568 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428159    2568 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428162    2568 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428166    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428169    2568 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428172    2568 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:07:53.432860 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428175    2568 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428177    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428180    2568 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428183    2568 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428185    2568 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428189    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428191    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428194    2568 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428197    2568 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428199    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428202    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428204    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428206    2568 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428209    2568 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428211    2568 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428214    2568 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428216    2568 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428218    2568 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428221    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428223    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:07:53.433315 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428225    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428228    2568 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428230    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428233    2568 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428235    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428237    2568 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428240    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428242    2568 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428245    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428247    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428249    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428252    2568 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428254    2568 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428257    2568 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428260    2568 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428262    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428265    2568 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428268    2568 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428271    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428273    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:07:53.433795 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428276    2568 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428279    2568 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428283    2568 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428287    2568 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428290    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428293    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428295    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428298    2568 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428300    2568 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428303    2568 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428305    2568 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428308    2568 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428310    2568 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428313    2568 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428315    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428318    2568 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428320    2568 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428323    2568 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428326    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:07:53.434289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428328    2568 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428330    2568 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428333    2568 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428335    2568 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428337    2568 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428340    2568 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428342    2568 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428346    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428348    2568 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428350    2568 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428353    2568 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428356    2568 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428359    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428361    2568 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428364    2568 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428366    2568 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428368    2568 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428371    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428373    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428376    2568 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:07:53.434782 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428378    2568 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:07:53.435325 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428381    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:07:53.435325 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428383    2568 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:07:53.435325 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.428411    2568 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:07:53.435325 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428543    2568 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:07:53.435325 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428547    2568 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:07:53.435325 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428550    2568 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:07:53.435325 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428553    2568 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:07:53.435325 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428555    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:07:53.435325 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428558    2568 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:07:53.435325 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428561    2568 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:07:53.435325 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428563    2568 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:07:53.435325 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428565    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:07:53.435325 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428568    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:07:53.435325 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428570    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:07:53.435325 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428573    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:07:53.435325 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428575    2568 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428578    2568 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428580    2568 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428583    2568 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428585    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428588    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428590    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428592    2568 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428595    2568 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428600    2568 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428603    2568 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428606    2568 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428609    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428611    2568 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428613    2568 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428616    2568 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428620    2568 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428624    2568 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428626    2568 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:07:53.435751 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428629    2568 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428631    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428634    2568 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428654    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428656    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428659    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428661    2568 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428664    2568 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428667    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428669    2568 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428672    2568 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428675    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428677    2568 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428680    2568 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428682    2568 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428684    2568 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428687    2568 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428690    2568 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428694    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:07:53.436184 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428697    2568 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428699    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428702    2568 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428704    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428707    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428710    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428713    2568 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428715    2568 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428717    2568 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428720    2568 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428722    2568 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428724    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428727    2568 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428729    2568 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428732    2568 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428735    2568 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428737    2568 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428740    2568 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428742    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428745    2568 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:07:53.436620 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428747    2568 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428750    2568 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428752    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428755    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428757    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428759    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428762    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428764    2568 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428767    2568 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428769    2568 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428771    2568 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428774    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428776    2568 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428779    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428781    2568 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428784    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428786    2568 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428789    2568 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428791    2568 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:07:53.437358 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428794    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:07:53.438019 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428796    2568 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:07:53.438019 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428799    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:07:53.438019 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428801    2568 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:07:53.438019 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428803    2568 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:07:53.438019 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428806    2568 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:07:53.438019 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428808    2568 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:07:53.438019 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428811    2568 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:07:53.438019 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428813    2568 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:07:53.438019 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428816    2568 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:07:53.438019 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428818    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:07:53.438019 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428820    2568 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:07:53.438019 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428823    2568 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:07:53.438019 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428825    2568 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:07:53.438019 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428828    2568 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:07:53.438019 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428830    2568 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:07:53.438019 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428833    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:07:53.438403 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.428855    2568 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:07:53.438403 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428891    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:07:53.438403 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428896    2568 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:07:53.438403 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428899    2568 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:07:53.438403 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428902    2568 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:07:53.438403 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428905    2568 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:07:53.438403 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428908    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:07:53.438403 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428910    2568 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:07:53.438403 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428913    2568 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:07:53.438403 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428915    2568 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:07:53.438403 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428918    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:07:53.438403 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428920    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:07:53.438403 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428922    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:07:53.438403 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428925    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:07:53.438403 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428931    2568 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:07:53.438403 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428934    2568 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428936    2568 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428938    2568 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428941    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428943    2568 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428946    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428948    2568 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428950    2568 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428953    2568 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428955    2568 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428958    2568 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428960    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428962    2568 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428965    2568 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428967    2568 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428973    2568 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428977    2568 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428980    2568 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428983    2568 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:07:53.439055 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428986    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428990    2568 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428992    2568 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428995    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.428997    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429000    2568 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429003    2568 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429006    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429008    2568 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429011    2568 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429014    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429016    2568 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429020    2568 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429023    2568 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429026    2568 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429028    2568 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429032    2568 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429036    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429039    2568 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:07:53.439548 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429042    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429045    2568 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429048    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429050    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429053    2568 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429056    2568 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429058    2568 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429061    2568 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429063    2568 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429067    2568 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429070    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429072    2568 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429075    2568 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429077    2568 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429080    2568 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429082    2568 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429085    2568 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429087    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429090    2568 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429092    2568 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:07:53.440189 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429095    2568 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429097    2568 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429100    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429102    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429105    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429110    2568 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429112    2568 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429115    2568 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429118    2568 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429121    2568 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429123    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429125    2568 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429128    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429130    2568 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429133    2568 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429135    2568 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429137    2568 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429140    2568 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429142    2568 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:07:53.440736 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429145    2568 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429148    2568 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429152    2568 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429155    2568 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429157    2568 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429160    2568 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429162    2568 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429172    2568 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429175    2568 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429177    2568 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429180    2568 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429183    2568 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429185    2568 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429188    2568 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:53.429190    2568 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.429240    2568 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.430956    2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.432069    2568 server.go:1014] "Starting client certificate rotation"
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.432165    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 22:07:53.441251 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.432597    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 22:07:53.441717 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.433990    2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 22:07:53.441717 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.435050    2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 22:07:53.446589 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.446573    2568 log.go:25] "Validated CRI v1 runtime API"
Oct 30 22:07:53.449601 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.449586    2568 log.go:25] "Validated CRI v1 image API"
Oct 30 22:07:53.450443 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.450419    2568 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 22:07:53.454150 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.454130    2568 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7f371b9b-78c0-479a-a134-4f741cdd6c33:/dev/nvme0n1p4 8b6e8eaf-c32f-46f2-a685-692c172b452b:/dev/nvme0n1p3]
Oct 30 22:07:53.454210 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.454150    2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 22:07:53.459156 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.459047    2568 manager.go:217] Machine: {Timestamp:2026-10-30 22:07:53.457752677 +0000 UTC m=+0.326869220 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097965 MemoryCapacity:32812023808 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26d9563c835fd26b953ed410f3e2ea SystemUUID:ec26d956-3c83-5fd2-6b95-3ed410f3e2ea BootID:b44acf37-14d0-44a4-8170-5f40994a9b6d Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406011904 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0a:44:f2:1c:59 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0a:44:f2:1c:59 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:f8:ee:60:a4:75 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812023808 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 22:07:53.459800 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.459790    2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 22:07:53.459904 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.459892    2568 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 22:07:53.462090 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.462063    2568 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 22:07:53.462224 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.462092    2568 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-202.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 22:07:53.462271 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.462230    2568 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 22:07:53.462271 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.462238    2568 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 22:07:53.462325 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.462271    2568 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 22:07:53.462948 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.462938    2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 22:07:53.463594 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.463585    2568 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 22:07:53.467323 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.467311    2568 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 22:07:53.467429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.467421    2568 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 22:07:53.468303 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.468294    2568 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 22:07:53.468340 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.468306    2568 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 22:07:53.468340 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.468315    2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 22:07:53.468340 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.468324    2568 kubelet.go:405] "Adding apiserver pod source"
Oct 30 22:07:53.468340 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.468338    2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 22:07:53.468622 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.468610    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 22:07:53.470975 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.470953    2568 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 22:07:53.475798 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.475783    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 22:07:53.476728 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.476713    2568 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 22:07:53.476774 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.476749    2568 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 22:07:53.476910 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.476899    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 22:07:53.476944 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.476916    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 22:07:53.476944 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.476923    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 22:07:53.476944 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.476929    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 22:07:53.476944 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.476934    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 22:07:53.476944 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.476940    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 22:07:53.477540 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.477531    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 22:07:53.477569 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.477540    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 22:07:53.477569 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.477548    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 22:07:53.477569 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.477554    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 22:07:53.477569 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.477562    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 22:07:53.477823 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.477572    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 22:07:53.478723 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.478714    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 22:07:53.478723 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.478724    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 22:07:53.482254 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.482241    2568 server.go:1263] "Started kubelet"
Oct 30 22:07:53.482349 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.482317    2568 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 22:07:53.482404 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.482336    2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 22:07:53.482451 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.482423    2568 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 22:07:53.482799 ip-10-0-138-202 systemd[1]: Started Kubernetes Kubelet.
Oct 30 22:07:53.486888 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.486865    2568 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 22:07:53.487815 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.487798    2568 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 22:07:53.488085 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:53.488066    2568 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 22:07:53.490686 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.490669    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 22:07:53.491152 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.491138    2568 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 22:07:53.491547 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:53.491529    2568 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-202.us-west-1.compute.internal\" not found"
Oct 30 22:07:53.491927 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.491904    2568 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 22:07:53.491927 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.491926    2568 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 22:07:53.492097 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.492013    2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 22:07:53.492097 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.492067    2568 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 22:07:53.492097 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.492074    2568 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 22:07:53.492253 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.492237    2568 factory.go:55] Registering systemd factory
Oct 30 22:07:53.492298 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.492266    2568 factory.go:223] Registration of the systemd container factory successfully
Oct 30 22:07:53.492474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.492456    2568 factory.go:153] Registering CRI-O factory
Oct 30 22:07:53.492562 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.492477    2568 factory.go:223] Registration of the crio container factory successfully
Oct 30 22:07:53.492562 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.492528    2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 22:07:53.492562 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.492546    2568 factory.go:103] Registering Raw factory
Oct 30 22:07:53.492562 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.492562    2568 manager.go:1199] Started watching for new ooms in manager
Oct 30 22:07:53.493363 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.493350    2568 manager.go:319] Starting recovery of all containers
Oct 30 22:07:53.503158 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.503146    2568 manager.go:324] Recovery completed
Oct 30 22:07:53.506991 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.506979    2568 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:07:53.508788 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.508773    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:07:53.508855 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.508802    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:07:53.508855 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.508813    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:07:53.509294 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.509279    2568 cpu_manager.go:226] "Starting" policy="none"
Oct 30 22:07:53.509294 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.509287    2568 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:07:53.509294 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.509292    2568 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 22:07:53.509432 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.509311    2568 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 22:07:53.510045 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.510031    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:07:53.510111 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.510060    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:07:53.510111 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.510076    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:07:53.511548 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.511536    2568 policy_none.go:50] "Start"
Oct 30 22:07:53.511596 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.511551    2568 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 22:07:53.511596 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.511568    2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 22:07:53.513532 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.513521    2568 policy_none.go:44] "Start"
Oct 30 22:07:53.548194 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.548159    2568 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 22:07:53.551907 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.551759    2568 manager.go:341] "Starting Device Plugin manager"
Oct 30 22:07:53.551992 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:53.551932    2568 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 22:07:53.551992 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.551946    2568 server.go:87] "Starting device plugin registration server"
Oct 30 22:07:53.552210 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.552185    2568 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 22:07:53.552266 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.552203    2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 22:07:53.552266 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.552259    2568 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 22:07:53.552361 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.552352    2568 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 22:07:53.552407 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.552361    2568 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 22:07:53.552793 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:53.552773    2568 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 22:07:53.552875 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:53.552816    2568 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-202.us-west-1.compute.internal\" not found"
Oct 30 22:07:53.626824 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.626798    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 22:07:53.628500 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.628121    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 22:07:53.628500 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.628137    2568 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 22:07:53.628500 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.628151    2568 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 22:07:53.628500 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:53.628184    2568 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 22:07:53.652284 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.652265    2568 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:07:53.652946 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.652928    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:07:53.653047 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.652964    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:07:53.653047 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.652975    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:07:53.653047 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.652995    2568 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:53.684813 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:53.684777    2568 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-202.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 22:07:53.684813 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.684799    2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-202.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 22:07:53.685601 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:53.684868    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-202.us-west-1.compute.internal.18e36e0bc6c6f062  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-202.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-202.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 22:07:53.482211426 +0000 UTC m=+0.351327970,LastTimestamp:2026-10-30 22:07:53.482211426 +0000 UTC m=+0.351327970,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-202.us-west-1.compute.internal,}"
Oct 30 22:07:53.729218 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.729196    2568 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal"]
Oct 30 22:07:53.729288 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.729242    2568 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:07:53.729988 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.729974    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:07:53.730058 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.730000    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:07:53.730058 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.730010    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:07:53.731977 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.731966    2568 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:07:53.732116 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.732100    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:53.732160 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.732130    2568 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:07:53.732652 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.732630    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:07:53.732747 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.732656    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:07:53.732747 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.732665    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:07:53.732747 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.732677    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:07:53.732747 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.732678    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:07:53.732747 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.732743    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:07:53.734418 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.734403    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:53.734488 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.734426    2568 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:07:53.735135 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.735121    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:07:53.735202 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.735147    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:07:53.735202 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.735156    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:07:53.737969 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.737953    2568 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 22:07:53.738053 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.738006    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sscxn"
Oct 30 22:07:53.757605 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:53.757588    2568 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-202.us-west-1.compute.internal\" not found" node="ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:53.759484 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:53.759469    2568 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-202.us-west-1.compute.internal\" not found" node="ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:53.796610 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.796594    2568 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:07:53.844481 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.844464    2568 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:07:53.893691 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.893671    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2c344a83145e337d085fbb4af4282fc3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"2c344a83145e337d085fbb4af4282fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:53.893771 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.893706    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c344a83145e337d085fbb4af4282fc3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"2c344a83145e337d085fbb4af4282fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:53.893771 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.893737    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc53250de6efc08206be074330c1cea0-config\") pod \"kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"dc53250de6efc08206be074330c1cea0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:53.894315 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.894292    2568 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:07:53.894927 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.894912    2568 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:07:53.895789 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.895773    2568 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:53.895864 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:53.895793    2568 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-202.us-west-1.compute.internal\": node \"ip-10-0-138-202.us-west-1.compute.internal\" not found"
Oct 30 22:07:53.993087 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.993042    2568 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:53.994174 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.994159    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2c344a83145e337d085fbb4af4282fc3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"2c344a83145e337d085fbb4af4282fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:53.994233 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.994183    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c344a83145e337d085fbb4af4282fc3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"2c344a83145e337d085fbb4af4282fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:53.994233 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.994201    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc53250de6efc08206be074330c1cea0-config\") pod \"kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"dc53250de6efc08206be074330c1cea0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:53.994327 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.994254    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc53250de6efc08206be074330c1cea0-config\") pod \"kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"dc53250de6efc08206be074330c1cea0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:53.994327 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.994277    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c344a83145e337d085fbb4af4282fc3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"2c344a83145e337d085fbb4af4282fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:53.994327 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:53.994277    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2c344a83145e337d085fbb4af4282fc3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"2c344a83145e337d085fbb4af4282fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:54.058404 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.058384    2568 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:07:54.058491 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.058476    2568 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:54.060539 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.060526    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:54.062704 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.062690    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:07:54.116325 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.116303    2568 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:07:54.431548 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.431482    2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 22:07:54.431999 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.431602    2568 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 22:07:54.431999 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.431627    2568 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 22:07:54.431999 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.431603    2568 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 22:07:54.431999 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:54.431656    2568 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a6f4d41d4389f4fb28595b18760ef362-f326b7236f2e3088.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-138-202.us-west-1.compute.internal.18e36e0bc85cea9d\": read tcp 10.0.138.202:39602->3.130.97.254:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-138-202.us-west-1.compute.internal.18e36e0bc85cea9d  default   5291 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-202.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-202.us-west-1.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-202.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 22:07:53 +0000 UTC,LastTimestamp:2026-10-30 22:07:53.652979186 +0000 UTC m=+0.522095730,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-202.us-west-1.compute.internal,}"
Oct 30 22:07:54.468463 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.468443    2568 apiserver.go:52] "Watching apiserver"
Oct 30 22:07:54.491649 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.491629    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 22:07:54.741889 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.741813    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 22:02:54 +0000 UTC" deadline="2028-04-20 13:35:48.70742253 +0000 UTC"
Oct 30 22:07:54.741889 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.741840    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12903h27m53.965584446s"
Oct 30 22:07:54.745184 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.745157    2568 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 22:07:54.745444 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.745424    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-lr7w8","openshift-network-operator/iptables-alerter-nb5dx","openshift-ovn-kubernetes/ovnkube-node-vcjjm","kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal","openshift-cluster-node-tuning-operator/tuned-57cq6","openshift-image-registry/node-ca-krfnw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal","openshift-multus/multus-additional-cni-plugins-pndld","openshift-multus/multus-cshfp","openshift-multus/network-metrics-daemon-fgbcv","openshift-network-diagnostics/network-check-target-s5qdj","kube-system/global-pull-secret-syncer-jxjd5","kube-system/konnectivity-agent-f4lgs","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"]
Oct 30 22:07:54.747792 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.747776    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:07:54.747884 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:54.747865    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:07:54.749757 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.749739    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:07:54.751823 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.751810    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.753862 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.753841    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.755830 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.755814    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.757822 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.757805    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:07:54.759912 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.759893    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.762012 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.761970    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:07:54.764161 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.764148    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:07:54.766273 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.766260    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:07:54.766332 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:54.766318    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:07:54.768447 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.768432    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:07:54.768509 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:54.768493    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:07:54.770559 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.770545    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.793036 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.793021    2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 22:07:54.797789 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.797765    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-cnibin\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.797874 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.797800    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-netns\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.797874 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.797825    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-cnibin\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.797985 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.797881    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-socket-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.797985 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.797937    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v8wz\" (UniqueName: \"kubernetes.io/projected/7c315adb-397e-4418-b193-74697cbd6b80-kube-api-access-6v8wz\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:07:54.797985 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.797976    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/515e0d50-7bf5-476d-a84b-79e5f21f959f-serviceca\") pod \"node-ca-krfnw\" (UID: \"515e0d50-7bf5-476d-a84b-79e5f21f959f\") " pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:07:54.798093 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798025    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-cni-bin\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.798093 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798052    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-system-cni-dir\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.798093 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798081    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-systemd\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.798217 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798118    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/653cd0d6-b7a2-446a-b677-406e3e8e56ce-agent-certs\") pod \"konnectivity-agent-f4lgs\" (UID: \"653cd0d6-b7a2-446a-b677-406e3e8e56ce\") " pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:07:54.798217 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798146    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-kubelet\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.798217 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798176    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.798300 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798221    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:07:54.798300 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798239    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-kubelet-config\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:07:54.798300 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798274    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.798382 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798316    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-ovn\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.798382 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798359    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovnkube-script-lib\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.798440 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798389    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/62af32b1-6029-4da7-b299-0d3ba5746dc6-hosts-file\") pod \"node-resolver-lr7w8\" (UID: \"62af32b1-6029-4da7-b299-0d3ba5746dc6\") " pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:07:54.798470 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798449    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-conf-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.798500 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798476    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgnz9\" (UniqueName: \"kubernetes.io/projected/62af32b1-6029-4da7-b299-0d3ba5746dc6-kube-api-access-lgnz9\") pod \"node-resolver-lr7w8\" (UID: \"62af32b1-6029-4da7-b299-0d3ba5746dc6\") " pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:07:54.798540 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798500    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/515e0d50-7bf5-476d-a84b-79e5f21f959f-host\") pod \"node-ca-krfnw\" (UID: \"515e0d50-7bf5-476d-a84b-79e5f21f959f\") " pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:07:54.798540 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798529    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljk86\" (UniqueName: \"kubernetes.io/projected/515e0d50-7bf5-476d-a84b-79e5f21f959f-kube-api-access-ljk86\") pod \"node-ca-krfnw\" (UID: \"515e0d50-7bf5-476d-a84b-79e5f21f959f\") " pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:07:54.798621 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798546    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f122e9a3-7599-48d2-b5fc-1d373d7cf446-cni-binary-copy\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.798621 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798598    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-dbus\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:07:54.798731 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798665    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.798731 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798706    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.798731 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798723    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-systemd-units\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.798870 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798740    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-node-log\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.798870 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798754    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f305ea8d-018d-4c14-a102-de41495bf1a9-tmp\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.798870 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798795    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-k8s-cni-cncf-io\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.798870 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798822    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-cni-multus\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.798870 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798866    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-hostroot\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.799073 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798897    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-kubelet\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.799073 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798918    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-cni-netd\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.799073 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798937    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62af32b1-6029-4da7-b299-0d3ba5746dc6-tmp-dir\") pod \"node-resolver-lr7w8\" (UID: \"62af32b1-6029-4da7-b299-0d3ba5746dc6\") " pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:07:54.799073 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798968    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-socket-dir-parent\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.799073 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.798986    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-log-socket\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.799073 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799015    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-cni-bin\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.799073 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799043    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-env-overrides\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.799073 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799059    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovn-node-metrics-cert\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.799474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799079    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf8fa689-675c-4153-a857-07894dba2fe6-iptables-alerter-script\") pod \"iptables-alerter-nb5dx\" (UID: \"cf8fa689-675c-4153-a857-07894dba2fe6\") " pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:07:54.799474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799097    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7dn7\" (UniqueName: \"kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7\") pod \"network-check-target-s5qdj\" (UID: \"a414991b-4e9e-42e7-be00-a2b460ea3b2b\") " pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:07:54.799474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799136    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysctl-conf\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.799474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799162    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:07:54.799474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799200    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mct7c\" (UniqueName: \"kubernetes.io/projected/66d962d8-e0a3-4c4e-b310-e19aa5360593-kube-api-access-mct7c\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.799474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799221    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-etc-selinux\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.799474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799271    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-sys-fs\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.799474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799307    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysconfig\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.799474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799346    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-kubernetes\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.799474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799369    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysctl-d\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.799474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799385    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-systemd\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.799474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799405    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-cni-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.799474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799432    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-cni-binary-copy\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.799474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799449    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-var-lib-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.799474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799467    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-etc-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.800209 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799490    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-run-ovn-kubernetes\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.800209 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799535    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf8fa689-675c-4153-a857-07894dba2fe6-host-slash\") pod \"iptables-alerter-nb5dx\" (UID: \"cf8fa689-675c-4153-a857-07894dba2fe6\") " pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:07:54.800209 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799569    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-modprobe-d\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.800209 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799607    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-run-netns\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.800209 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799655    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.800209 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799689    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovnkube-config\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.800209 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799714    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-registration-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.800209 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799733    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/653cd0d6-b7a2-446a-b677-406e3e8e56ce-konnectivity-ca\") pod \"konnectivity-agent-f4lgs\" (UID: \"653cd0d6-b7a2-446a-b677-406e3e8e56ce\") " pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:07:54.800209 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799756    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-var-lib-kubelet\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.800209 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799776    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-tuned\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.800209 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799798    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5zh5\" (UniqueName: \"kubernetes.io/projected/f305ea8d-018d-4c14-a102-de41495bf1a9-kube-api-access-m5zh5\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.800209 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799824    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-system-cni-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.800209 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799859    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-daemon-config\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.800209 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799881    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-etc-kubernetes\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.800209 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799934    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-run\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.800209 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799952    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-lib-modules\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.801062 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.799974    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-multus-certs\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.801062 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.800004    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46nlc\" (UniqueName: \"kubernetes.io/projected/f122e9a3-7599-48d2-b5fc-1d373d7cf446-kube-api-access-46nlc\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.801062 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.800021    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.801062 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.800058    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89c5l\" (UniqueName: \"kubernetes.io/projected/10c86ffc-dbfb-48d8-a225-edc41c719c6f-kube-api-access-89c5l\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.801062 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.800106    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-device-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.801062 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.800133    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bg9c\" (UniqueName: \"kubernetes.io/projected/cf8fa689-675c-4153-a857-07894dba2fe6-kube-api-access-2bg9c\") pod \"iptables-alerter-nb5dx\" (UID: \"cf8fa689-675c-4153-a857-07894dba2fe6\") " pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:07:54.801062 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.800165    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-sys\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.801062 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.800194    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-host\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.801062 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.800233    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-slash\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.801062 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.800273    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxwgg\" (UniqueName: \"kubernetes.io/projected/0f6f4765-3e5c-441e-a388-8f56a6c180a8-kube-api-access-xxwgg\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.815401 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.815387    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 22:07:54.815480 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.815387    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 22:07:54.815480 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.815432    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-2djd4\""
Oct 30 22:07:54.816184 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.816171    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lmcdt\""
Oct 30 22:07:54.819721 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.819705    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 22:07:54.819811 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.819798    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 22:07:54.899574 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.899549    2568 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 22:07:54.899665 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.899601    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mrr9c"
Oct 30 22:07:54.900660 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.900624    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-log-socket\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.900721 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.900668    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-cni-bin\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.900721 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.900717    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-cni-bin\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.900821 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.900720    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-log-socket\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.900821 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.900738    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-env-overrides\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.900821 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.900754    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovn-node-metrics-cert\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.900821 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.900768    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf8fa689-675c-4153-a857-07894dba2fe6-iptables-alerter-script\") pod \"iptables-alerter-nb5dx\" (UID: \"cf8fa689-675c-4153-a857-07894dba2fe6\") " pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:07:54.900821 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.900782    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7dn7\" (UniqueName: \"kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7\") pod \"network-check-target-s5qdj\" (UID: \"a414991b-4e9e-42e7-be00-a2b460ea3b2b\") " pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:07:54.900821 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.900796    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysctl-conf\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.900821 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.900809    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:07:54.901114 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.900826    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mct7c\" (UniqueName: \"kubernetes.io/projected/66d962d8-e0a3-4c4e-b310-e19aa5360593-kube-api-access-mct7c\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.901114 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:54.900933    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:07:54.901114 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.900973    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysctl-conf\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.901114 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:54.901003    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret podName:796116fb-6bb5-4c50-8b4f-9aabe94ea8ce nodeName:}" failed. No retries permitted until 2026-10-30 22:07:55.400980699 +0000 UTC m=+2.270097230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret") pod "global-pull-secret-syncer-jxjd5" (UID: "796116fb-6bb5-4c50-8b4f-9aabe94ea8ce") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:07:54.901114 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901028    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-etc-selinux\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.901114 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901069    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-sys-fs\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.901114 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901094    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysconfig\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901117    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-kubernetes\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901125    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-env-overrides\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901139    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysctl-d\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901145    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-etc-selinux\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901163    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-systemd\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901192    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-sys-fs\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901204    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-kubernetes\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901195    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysconfig\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901195    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-cni-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901239    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-systemd\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901249    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-cni-binary-copy\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901273    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysctl-d\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901274    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-var-lib-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901310    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-etc-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901334    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-run-ovn-kubernetes\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901338    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-etc-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.901429 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901351    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf8fa689-675c-4153-a857-07894dba2fe6-host-slash\") pod \"iptables-alerter-nb5dx\" (UID: \"cf8fa689-675c-4153-a857-07894dba2fe6\") " pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901365    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-modprobe-d\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901366    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-run-ovn-kubernetes\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901309    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-var-lib-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901312    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-cni-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901394    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-run-netns\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901413    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901418    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf8fa689-675c-4153-a857-07894dba2fe6-host-slash\") pod \"iptables-alerter-nb5dx\" (UID: \"cf8fa689-675c-4153-a857-07894dba2fe6\") " pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901428    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovnkube-config\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901435    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-modprobe-d\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901459    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-run-netns\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901456    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-registration-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901481    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901502    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/653cd0d6-b7a2-446a-b677-406e3e8e56ce-konnectivity-ca\") pod \"konnectivity-agent-f4lgs\" (UID: \"653cd0d6-b7a2-446a-b677-406e3e8e56ce\") " pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901518    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-var-lib-kubelet\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901532    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-tuned\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901545    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5zh5\" (UniqueName: \"kubernetes.io/projected/f305ea8d-018d-4c14-a102-de41495bf1a9-kube-api-access-m5zh5\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.902195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901560    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-system-cni-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901573    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-daemon-config\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901500    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-registration-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901587    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-etc-kubernetes\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901601    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-var-lib-kubelet\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901609    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-run\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901618    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-etc-kubernetes\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901632    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-lib-modules\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901674    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-system-cni-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901678    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-multus-certs\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901687    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-run\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901709    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46nlc\" (UniqueName: \"kubernetes.io/projected/f122e9a3-7599-48d2-b5fc-1d373d7cf446-kube-api-access-46nlc\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901714    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-multus-certs\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901734    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901749    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-lib-modules\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901758    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89c5l\" (UniqueName: \"kubernetes.io/projected/10c86ffc-dbfb-48d8-a225-edc41c719c6f-kube-api-access-89c5l\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901796    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901823    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-device-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.903003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901850    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bg9c\" (UniqueName: \"kubernetes.io/projected/cf8fa689-675c-4153-a857-07894dba2fe6-kube-api-access-2bg9c\") pod \"iptables-alerter-nb5dx\" (UID: \"cf8fa689-675c-4153-a857-07894dba2fe6\") " pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901872    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-sys\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901872    2568 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901893    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-host\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901914    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-slash\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901903    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-device-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901937    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxwgg\" (UniqueName: \"kubernetes.io/projected/0f6f4765-3e5c-441e-a388-8f56a6c180a8-kube-api-access-xxwgg\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901961    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-cnibin\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901964    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-slash\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901973    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-host\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901992    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-sys\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.901994    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-netns\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902032    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-netns\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902044    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/653cd0d6-b7a2-446a-b677-406e3e8e56ce-konnectivity-ca\") pod \"konnectivity-agent-f4lgs\" (UID: \"653cd0d6-b7a2-446a-b677-406e3e8e56ce\") " pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902051    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-cnibin\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902045    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-cnibin\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902074    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-cnibin\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902088    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-socket-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.903630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902110    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v8wz\" (UniqueName: \"kubernetes.io/projected/7c315adb-397e-4418-b193-74697cbd6b80-kube-api-access-6v8wz\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902126    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/515e0d50-7bf5-476d-a84b-79e5f21f959f-serviceca\") pod \"node-ca-krfnw\" (UID: \"515e0d50-7bf5-476d-a84b-79e5f21f959f\") " pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902141    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-cni-bin\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902163    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-system-cni-dir\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902185    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-systemd\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902206    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-socket-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902229    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-systemd\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902237    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-cni-bin\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902269    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-system-cni-dir\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902332    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/653cd0d6-b7a2-446a-b677-406e3e8e56ce-agent-certs\") pod \"konnectivity-agent-f4lgs\" (UID: \"653cd0d6-b7a2-446a-b677-406e3e8e56ce\") " pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902357    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-kubelet\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902378    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902402    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902421    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-kubelet-config\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902443    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902463    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-ovn\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902469    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.904134 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902483    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovnkube-script-lib\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902502    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/62af32b1-6029-4da7-b299-0d3ba5746dc6-hosts-file\") pod \"node-resolver-lr7w8\" (UID: \"62af32b1-6029-4da7-b299-0d3ba5746dc6\") " pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902515    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-kubelet\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902522    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-conf-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902563    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-conf-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902563    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-kubelet-config\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902583    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgnz9\" (UniqueName: \"kubernetes.io/projected/62af32b1-6029-4da7-b299-0d3ba5746dc6-kube-api-access-lgnz9\") pod \"node-resolver-lr7w8\" (UID: \"62af32b1-6029-4da7-b299-0d3ba5746dc6\") " pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:54.902503    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902609    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/515e0d50-7bf5-476d-a84b-79e5f21f959f-host\") pod \"node-ca-krfnw\" (UID: \"515e0d50-7bf5-476d-a84b-79e5f21f959f\") " pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902615    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/62af32b1-6029-4da7-b299-0d3ba5746dc6-hosts-file\") pod \"node-resolver-lr7w8\" (UID: \"62af32b1-6029-4da7-b299-0d3ba5746dc6\") " pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902653    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljk86\" (UniqueName: \"kubernetes.io/projected/515e0d50-7bf5-476d-a84b-79e5f21f959f-kube-api-access-ljk86\") pod \"node-ca-krfnw\" (UID: \"515e0d50-7bf5-476d-a84b-79e5f21f959f\") " pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902679    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f122e9a3-7599-48d2-b5fc-1d373d7cf446-cni-binary-copy\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902683    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/515e0d50-7bf5-476d-a84b-79e5f21f959f-host\") pod \"node-ca-krfnw\" (UID: \"515e0d50-7bf5-476d-a84b-79e5f21f959f\") " pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902600    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-ovn\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902703    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-dbus\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902727    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:54.902741    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs podName:7c315adb-397e-4418-b193-74697cbd6b80 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:55.402723088 +0000 UTC m=+2.271839634 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs") pod "network-metrics-daemon-fgbcv" (UID: "7c315adb-397e-4418-b193-74697cbd6b80") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:07:54.904575 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902778    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902808    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-systemd-units\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902833    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-node-log\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902859    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f305ea8d-018d-4c14-a102-de41495bf1a9-tmp\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902883    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902891    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-systemd-units\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902933    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-k8s-cni-cncf-io\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902860    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-dbus\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902885    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-k8s-cni-cncf-io\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.902970    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-node-log\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.903011    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-cni-multus\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.903039    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-hostroot\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.903055    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-kubelet\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.903061    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-cni-multus\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.903071    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-cni-netd\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.903111    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-cni-netd\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.903120    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62af32b1-6029-4da7-b299-0d3ba5746dc6-tmp-dir\") pod \"node-resolver-lr7w8\" (UID: \"62af32b1-6029-4da7-b299-0d3ba5746dc6\") " pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.903141    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-kubelet\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.905262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.903147    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-socket-dir-parent\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.906163 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.903122    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-hostroot\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.906163 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.903190    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-socket-dir-parent\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.906163 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.903383    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62af32b1-6029-4da7-b299-0d3ba5746dc6-tmp-dir\") pod \"node-resolver-lr7w8\" (UID: \"62af32b1-6029-4da7-b299-0d3ba5746dc6\") " pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:07:54.906163 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.905003    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-tuned\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.906163 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.905075    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f305ea8d-018d-4c14-a102-de41495bf1a9-tmp\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.906163 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.905295    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/653cd0d6-b7a2-446a-b677-406e3e8e56ce-agent-certs\") pod \"konnectivity-agent-f4lgs\" (UID: \"653cd0d6-b7a2-446a-b677-406e3e8e56ce\") " pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:07:54.915629 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.915598    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 22:07:54.915629 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.915604    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 22:07:54.915848 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.915658    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 22:07:54.915848 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.915715    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 22:07:54.915848 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.915752    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 22:07:54.915970 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.915909    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-486qv\""
Oct 30 22:07:54.915970 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.915946    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 22:07:54.916218 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916194    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 22:07:54.916218 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916196    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-78xqf\""
Oct 30 22:07:54.916218 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916216    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tmrfp\""
Oct 30 22:07:54.916483 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916194    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 22:07:54.916483 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916210    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dl5xt\""
Oct 30 22:07:54.916483 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916306    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 22:07:54.916483 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916380    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 22:07:54.916483 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916393    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 22:07:54.916483 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916395    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 22:07:54.916483 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916414    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 22:07:54.916483 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916422    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 22:07:54.916483 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916443    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-mmvq7\""
Oct 30 22:07:54.916483 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916450    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 22:07:54.916483 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916461    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 22:07:54.916483 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916470    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 22:07:54.916483 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916466    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 22:07:54.916483 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916481    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 22:07:54.916895 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.916527    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 22:07:54.922044 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.922025    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovnkube-config\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.922127 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.922049    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-cni-binary-copy\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.922127 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.922100    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-daemon-config\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.923784 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.923749    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/515e0d50-7bf5-476d-a84b-79e5f21f959f-serviceca\") pod \"node-ca-krfnw\" (UID: \"515e0d50-7bf5-476d-a84b-79e5f21f959f\") " pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:07:54.923784 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.923753    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.923940 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.923787    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f122e9a3-7599-48d2-b5fc-1d373d7cf446-cni-binary-copy\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.923940 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.923792    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovn-node-metrics-cert\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.923940 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.923815    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.965543 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.965515    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lds9w\""
Oct 30 22:07:54.965843 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.965813    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 22:07:54.965843 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.965823    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 22:07:54.966000 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.965862    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 22:07:54.966000 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:54.965947    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:07:54.966000 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.965953    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2wgjn\""
Oct 30 22:07:54.966000 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:54.965966    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:07:54.966000 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:54.965979    2568 projected.go:196] Error preparing data for projected volume kube-api-access-b7dn7 for pod openshift-network-diagnostics/network-check-target-s5qdj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:07:54.966182 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:54.966037    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7 podName:a414991b-4e9e-42e7-be00-a2b460ea3b2b nodeName:}" failed. No retries permitted until 2026-10-30 22:07:55.466018552 +0000 UTC m=+2.335135098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b7dn7" (UniqueName: "kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7") pod "network-check-target-s5qdj" (UID: "a414991b-4e9e-42e7-be00-a2b460ea3b2b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:07:54.970014 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.969987    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bg9c\" (UniqueName: \"kubernetes.io/projected/cf8fa689-675c-4153-a857-07894dba2fe6-kube-api-access-2bg9c\") pod \"iptables-alerter-nb5dx\" (UID: \"cf8fa689-675c-4153-a857-07894dba2fe6\") " pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:07:54.970014 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.970005    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxwgg\" (UniqueName: \"kubernetes.io/projected/0f6f4765-3e5c-441e-a388-8f56a6c180a8-kube-api-access-xxwgg\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:54.970181 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.970070    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v8wz\" (UniqueName: \"kubernetes.io/projected/7c315adb-397e-4418-b193-74697cbd6b80-kube-api-access-6v8wz\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:07:54.970181 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.970069    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89c5l\" (UniqueName: \"kubernetes.io/projected/10c86ffc-dbfb-48d8-a225-edc41c719c6f-kube-api-access-89c5l\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:54.970408 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.970392    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5zh5\" (UniqueName: \"kubernetes.io/projected/f305ea8d-018d-4c14-a102-de41495bf1a9-kube-api-access-m5zh5\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:54.970468 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.970447    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mct7c\" (UniqueName: \"kubernetes.io/projected/66d962d8-e0a3-4c4e-b310-e19aa5360593-kube-api-access-mct7c\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:54.970519 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.970472    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljk86\" (UniqueName: \"kubernetes.io/projected/515e0d50-7bf5-476d-a84b-79e5f21f959f-kube-api-access-ljk86\") pod \"node-ca-krfnw\" (UID: \"515e0d50-7bf5-476d-a84b-79e5f21f959f\") " pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:07:54.970519 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.970479    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46nlc\" (UniqueName: \"kubernetes.io/projected/f122e9a3-7599-48d2-b5fc-1d373d7cf446-kube-api-access-46nlc\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:07:54.970956 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.970938    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgnz9\" (UniqueName: \"kubernetes.io/projected/62af32b1-6029-4da7-b299-0d3ba5746dc6-kube-api-access-lgnz9\") pod \"node-resolver-lr7w8\" (UID: \"62af32b1-6029-4da7-b299-0d3ba5746dc6\") " pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:07:54.972081 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.972064    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf8fa689-675c-4153-a857-07894dba2fe6-iptables-alerter-script\") pod \"iptables-alerter-nb5dx\" (UID: \"cf8fa689-675c-4153-a857-07894dba2fe6\") " pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:07:54.973602 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:54.973584    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovnkube-script-lib\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:55.082439 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.082370    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:07:55.102461 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.102437    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:07:55.108293 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.107839    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:07:55.123740 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.123714    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:07:55.136900 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.136878    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:07:55.153862 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.153837    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cshfp"
Oct 30 22:07:55.162124 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.162109    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:07:55.187754 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.187729    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:07:55.204706 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.204681    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:07:55.221824 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:55.221802    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66d962d8_e0a3_4c4e_b310_e19aa5360593.slice/crio-fc46b1d3dc62cfa9bde50dc66e27f1d33f92338594cd4d548c722369ac5937b2 WatchSource:0}: Error finding container fc46b1d3dc62cfa9bde50dc66e27f1d33f92338594cd4d548c722369ac5937b2: Status 404 returned error can't find the container with id fc46b1d3dc62cfa9bde50dc66e27f1d33f92338594cd4d548c722369ac5937b2
Oct 30 22:07:55.222289 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:55.222266    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc53250de6efc08206be074330c1cea0.slice/crio-eb7a959b006264436cfb665fe0dc8c1d95f5708fe651d147affac90525cdb86c WatchSource:0}: Error finding container eb7a959b006264436cfb665fe0dc8c1d95f5708fe651d147affac90525cdb86c: Status 404 returned error can't find the container with id eb7a959b006264436cfb665fe0dc8c1d95f5708fe651d147affac90525cdb86c
Oct 30 22:07:55.223106 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:55.223085    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c344a83145e337d085fbb4af4282fc3.slice/crio-c63c3353e20169010c60f5b5a591841e10c90a80d30f9c47821c7d32c763fed9 WatchSource:0}: Error finding container c63c3353e20169010c60f5b5a591841e10c90a80d30f9c47821c7d32c763fed9: Status 404 returned error can't find the container with id c63c3353e20169010c60f5b5a591841e10c90a80d30f9c47821c7d32c763fed9
Oct 30 22:07:55.228836 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.228818    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:07:55.408218 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.408143    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:07:55.408218 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.408192    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:07:55.408384 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:55.408293    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:07:55.408384 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:55.408293    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:07:55.408384 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:55.408352    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs podName:7c315adb-397e-4418-b193-74697cbd6b80 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:56.40833319 +0000 UTC m=+3.277449722 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs") pod "network-metrics-daemon-fgbcv" (UID: "7c315adb-397e-4418-b193-74697cbd6b80") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:07:55.408384 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:55.408365    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret podName:796116fb-6bb5-4c50-8b4f-9aabe94ea8ce nodeName:}" failed. No retries permitted until 2026-10-30 22:07:56.40835889 +0000 UTC m=+3.277475420 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret") pod "global-pull-secret-syncer-jxjd5" (UID: "796116fb-6bb5-4c50-8b4f-9aabe94ea8ce") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:07:55.503038 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.503015    2568 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:07:55.509285 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.509264    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7dn7\" (UniqueName: \"kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7\") pod \"network-check-target-s5qdj\" (UID: \"a414991b-4e9e-42e7-be00-a2b460ea3b2b\") " pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:07:55.509366 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:55.509355    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:07:55.509402 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:55.509370    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:07:55.509402 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:55.509379    2568 projected.go:196] Error preparing data for projected volume kube-api-access-b7dn7 for pod openshift-network-diagnostics/network-check-target-s5qdj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:07:55.509460 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:55.509417    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7 podName:a414991b-4e9e-42e7-be00-a2b460ea3b2b nodeName:}" failed. No retries permitted until 2026-10-30 22:07:56.509406404 +0000 UTC m=+3.378522935 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7dn7" (UniqueName: "kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7") pod "network-check-target-s5qdj" (UID: "a414991b-4e9e-42e7-be00-a2b460ea3b2b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:07:55.607037 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.607013    2568 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:07:55.632790 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.632748    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pndld" event={"ID":"66d962d8-e0a3-4c4e-b310-e19aa5360593","Type":"ContainerStarted","Data":"fc46b1d3dc62cfa9bde50dc66e27f1d33f92338594cd4d548c722369ac5937b2"}
Oct 30 22:07:55.633813 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.633794    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal" event={"ID":"dc53250de6efc08206be074330c1cea0","Type":"ContainerStarted","Data":"eb7a959b006264436cfb665fe0dc8c1d95f5708fe651d147affac90525cdb86c"}
Oct 30 22:07:55.634874 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.634850    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal" event={"ID":"2c344a83145e337d085fbb4af4282fc3","Type":"ContainerStarted","Data":"c63c3353e20169010c60f5b5a591841e10c90a80d30f9c47821c7d32c763fed9"}
Oct 30 22:07:55.684198 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.684145    2568 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:07:55.891226 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:55.891199    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod653cd0d6_b7a2_446a_b677_406e3e8e56ce.slice/crio-88936199bf332ffa78567925b302310cff7b32634bfa61ef0ce08d322819056b WatchSource:0}: Error finding container 88936199bf332ffa78567925b302310cff7b32634bfa61ef0ce08d322819056b: Status 404 returned error can't find the container with id 88936199bf332ffa78567925b302310cff7b32634bfa61ef0ce08d322819056b
Oct 30 22:07:55.902696 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.902655    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 22:02:55 +0000 UTC" deadline="2028-08-01 21:12:06.22322294 +0000 UTC"
Oct 30 22:07:55.902696 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:55.902693    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15383h4m10.320532218s"
Oct 30 22:07:55.958130 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:55.958096    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10c86ffc_dbfb_48d8_a225_edc41c719c6f.slice/crio-960b2d65f6ce6ab539de3b3e43c196062928b3c35558d96550ae8ba45854fc6a WatchSource:0}: Error finding container 960b2d65f6ce6ab539de3b3e43c196062928b3c35558d96550ae8ba45854fc6a: Status 404 returned error can't find the container with id 960b2d65f6ce6ab539de3b3e43c196062928b3c35558d96550ae8ba45854fc6a
Oct 30 22:07:55.958522 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:55.958497    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f6f4765_3e5c_441e_a388_8f56a6c180a8.slice/crio-43a9d1f849e1d1cac670ea77815734593a4bc933243c71535cb9504330a0ebc8 WatchSource:0}: Error finding container 43a9d1f849e1d1cac670ea77815734593a4bc933243c71535cb9504330a0ebc8: Status 404 returned error can't find the container with id 43a9d1f849e1d1cac670ea77815734593a4bc933243c71535cb9504330a0ebc8
Oct 30 22:07:55.982985 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:55.982962    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf8fa689_675c_4153_a857_07894dba2fe6.slice/crio-4a1d5c898a7bc9ead0f87d7e30de621dba5ea85c36aa89a107ba490375462c36 WatchSource:0}: Error finding container 4a1d5c898a7bc9ead0f87d7e30de621dba5ea85c36aa89a107ba490375462c36: Status 404 returned error can't find the container with id 4a1d5c898a7bc9ead0f87d7e30de621dba5ea85c36aa89a107ba490375462c36
Oct 30 22:07:56.052433 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:56.052405    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf305ea8d_018d_4c14_a102_de41495bf1a9.slice/crio-40d58a800e323931d6a962f8e984cb3c55cff85a98e649e09cacf489fab8bee2 WatchSource:0}: Error finding container 40d58a800e323931d6a962f8e984cb3c55cff85a98e649e09cacf489fab8bee2: Status 404 returned error can't find the container with id 40d58a800e323931d6a962f8e984cb3c55cff85a98e649e09cacf489fab8bee2
Oct 30 22:07:56.053144 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:56.053119    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf122e9a3_7599_48d2_b5fc_1d373d7cf446.slice/crio-fc69fe88a9799c1453ea823b86041b89808fb2d7690c24d6575b31ce67652821 WatchSource:0}: Error finding container fc69fe88a9799c1453ea823b86041b89808fb2d7690c24d6575b31ce67652821: Status 404 returned error can't find the container with id fc69fe88a9799c1453ea823b86041b89808fb2d7690c24d6575b31ce67652821
Oct 30 22:07:56.077694 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:56.077675    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod515e0d50_7bf5_476d_a84b_79e5f21f959f.slice/crio-3c8636df0f9c440a3e0a695864d0ae42f268c4f81bbec6c28f2d73a30e375eb5 WatchSource:0}: Error finding container 3c8636df0f9c440a3e0a695864d0ae42f268c4f81bbec6c28f2d73a30e375eb5: Status 404 returned error can't find the container with id 3c8636df0f9c440a3e0a695864d0ae42f268c4f81bbec6c28f2d73a30e375eb5
Oct 30 22:07:56.167777 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:07:56.167756    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62af32b1_6029_4da7_b299_0d3ba5746dc6.slice/crio-805374b844f5cde99fd270dd7a7fc0fe4ad77396cdc9175d9b6bb73195eeffb3 WatchSource:0}: Error finding container 805374b844f5cde99fd270dd7a7fc0fe4ad77396cdc9175d9b6bb73195eeffb3: Status 404 returned error can't find the container with id 805374b844f5cde99fd270dd7a7fc0fe4ad77396cdc9175d9b6bb73195eeffb3
Oct 30 22:07:56.416195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:56.416116    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:07:56.416195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:56.416180    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:07:56.416416 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:56.416288    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:07:56.416416 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:56.416349    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret podName:796116fb-6bb5-4c50-8b4f-9aabe94ea8ce nodeName:}" failed. No retries permitted until 2026-10-30 22:07:58.416328844 +0000 UTC m=+5.285445391 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret") pod "global-pull-secret-syncer-jxjd5" (UID: "796116fb-6bb5-4c50-8b4f-9aabe94ea8ce") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:07:56.416416 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:56.416416    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:07:56.416550 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:56.416455    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs podName:7c315adb-397e-4418-b193-74697cbd6b80 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:58.416441381 +0000 UTC m=+5.285557930 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs") pod "network-metrics-daemon-fgbcv" (UID: "7c315adb-397e-4418-b193-74697cbd6b80") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:07:56.517148 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:56.517118    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7dn7\" (UniqueName: \"kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7\") pod \"network-check-target-s5qdj\" (UID: \"a414991b-4e9e-42e7-be00-a2b460ea3b2b\") " pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:07:56.517795 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:56.517282    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:07:56.517795 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:56.517301    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:07:56.517795 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:56.517314    2568 projected.go:196] Error preparing data for projected volume kube-api-access-b7dn7 for pod openshift-network-diagnostics/network-check-target-s5qdj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:07:56.517795 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:56.517369    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7 podName:a414991b-4e9e-42e7-be00-a2b460ea3b2b nodeName:}" failed. No retries permitted until 2026-10-30 22:07:58.517351613 +0000 UTC m=+5.386468154 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7dn7" (UniqueName: "kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7") pod "network-check-target-s5qdj" (UID: "a414991b-4e9e-42e7-be00-a2b460ea3b2b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:07:56.629409 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:56.629375    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:07:56.629580 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:56.629514    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:07:56.629704 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:56.629622    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:07:56.629774 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:56.629735    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:07:56.629837 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:56.629829    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:07:56.629974 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:56.629903    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:07:56.639851 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:56.639814    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm" event={"ID":"10c86ffc-dbfb-48d8-a225-edc41c719c6f","Type":"ContainerStarted","Data":"960b2d65f6ce6ab539de3b3e43c196062928b3c35558d96550ae8ba45854fc6a"}
Oct 30 22:07:56.641615 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:56.641589    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-f4lgs" event={"ID":"653cd0d6-b7a2-446a-b677-406e3e8e56ce","Type":"ContainerStarted","Data":"88936199bf332ffa78567925b302310cff7b32634bfa61ef0ce08d322819056b"}
Oct 30 22:07:56.643668 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:56.643624    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-krfnw" event={"ID":"515e0d50-7bf5-476d-a84b-79e5f21f959f","Type":"ContainerStarted","Data":"3c8636df0f9c440a3e0a695864d0ae42f268c4f81bbec6c28f2d73a30e375eb5"}
Oct 30 22:07:56.645956 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:56.645887    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-57cq6" event={"ID":"f305ea8d-018d-4c14-a102-de41495bf1a9","Type":"ContainerStarted","Data":"40d58a800e323931d6a962f8e984cb3c55cff85a98e649e09cacf489fab8bee2"}
Oct 30 22:07:56.647501 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:56.647476    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nb5dx" event={"ID":"cf8fa689-675c-4153-a857-07894dba2fe6","Type":"ContainerStarted","Data":"4a1d5c898a7bc9ead0f87d7e30de621dba5ea85c36aa89a107ba490375462c36"}
Oct 30 22:07:56.649131 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:56.649103    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lr7w8" event={"ID":"62af32b1-6029-4da7-b299-0d3ba5746dc6","Type":"ContainerStarted","Data":"805374b844f5cde99fd270dd7a7fc0fe4ad77396cdc9175d9b6bb73195eeffb3"}
Oct 30 22:07:56.656964 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:56.656901    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cshfp" event={"ID":"f122e9a3-7599-48d2-b5fc-1d373d7cf446","Type":"ContainerStarted","Data":"fc69fe88a9799c1453ea823b86041b89808fb2d7690c24d6575b31ce67652821"}
Oct 30 22:07:56.660696 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:56.660666    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j" event={"ID":"0f6f4765-3e5c-441e-a388-8f56a6c180a8","Type":"ContainerStarted","Data":"43a9d1f849e1d1cac670ea77815734593a4bc933243c71535cb9504330a0ebc8"}
Oct 30 22:07:56.902998 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:56.902881    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 22:02:55 +0000 UTC" deadline="2028-06-27 01:57:19.651491076 +0000 UTC"
Oct 30 22:07:56.902998 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:56.902953    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14523h49m22.748543019s"
Oct 30 22:07:58.437235 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:58.436611    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:07:58.437235 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:58.436703    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:07:58.437235 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:58.436847    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:07:58.437235 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:58.436915    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs podName:7c315adb-397e-4418-b193-74697cbd6b80 nodeName:}" failed. No retries permitted until 2026-10-30 22:08:02.436896023 +0000 UTC m=+9.306012557 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs") pod "network-metrics-daemon-fgbcv" (UID: "7c315adb-397e-4418-b193-74697cbd6b80") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:07:58.437235 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:58.436979    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:07:58.437235 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:58.437020    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret podName:796116fb-6bb5-4c50-8b4f-9aabe94ea8ce nodeName:}" failed. No retries permitted until 2026-10-30 22:08:02.43700568 +0000 UTC m=+9.306122216 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret") pod "global-pull-secret-syncer-jxjd5" (UID: "796116fb-6bb5-4c50-8b4f-9aabe94ea8ce") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:07:58.538936 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:58.537926    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7dn7\" (UniqueName: \"kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7\") pod \"network-check-target-s5qdj\" (UID: \"a414991b-4e9e-42e7-be00-a2b460ea3b2b\") " pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:07:58.538936 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:58.538157    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:07:58.538936 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:58.538177    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:07:58.538936 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:58.538190    2568 projected.go:196] Error preparing data for projected volume kube-api-access-b7dn7 for pod openshift-network-diagnostics/network-check-target-s5qdj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:07:58.538936 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:58.538248    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7 podName:a414991b-4e9e-42e7-be00-a2b460ea3b2b nodeName:}" failed. No retries permitted until 2026-10-30 22:08:02.538228945 +0000 UTC m=+9.407345508 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7dn7" (UniqueName: "kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7") pod "network-check-target-s5qdj" (UID: "a414991b-4e9e-42e7-be00-a2b460ea3b2b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:07:58.629370 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:58.628889    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:07:58.629370 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:58.629030    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:07:58.629370 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:58.629108    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:07:58.629370 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:58.629184    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:07:58.629370 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:58.629248    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:07:58.629370 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:07:58.629330    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:07:59.712362 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:59.712272    2568 generic.go:356] "Generic (PLEG): container finished" podID="66d962d8-e0a3-4c4e-b310-e19aa5360593" containerID="8c04266661e18a2ea0225a5660bde5c328c2ae8de13417c2d227fd24a051e77a" exitCode=0
Oct 30 22:07:59.713033 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:59.712940    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pndld" event={"ID":"66d962d8-e0a3-4c4e-b310-e19aa5360593","Type":"ContainerDied","Data":"8c04266661e18a2ea0225a5660bde5c328c2ae8de13417c2d227fd24a051e77a"}
Oct 30 22:07:59.731683 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:59.731585    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal" event={"ID":"dc53250de6efc08206be074330c1cea0","Type":"ContainerStarted","Data":"7840ffd6b583d319db66e45161f07717f90729ef5c06a1f4fa24c0fa7fa5f125"}
Oct 30 22:07:59.735810 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:59.735207    2568 generic.go:356] "Generic (PLEG): container finished" podID="2c344a83145e337d085fbb4af4282fc3" containerID="d5365a3119f96fcaae660bc5060c9c6890d4e467c48d5fcaa4dcfbadb6727e2d" exitCode=0
Oct 30 22:07:59.735810 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:07:59.735248    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal" event={"ID":"2c344a83145e337d085fbb4af4282fc3","Type":"ContainerDied","Data":"d5365a3119f96fcaae660bc5060c9c6890d4e467c48d5fcaa4dcfbadb6727e2d"}
Oct 30 22:08:00.109952 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:00.109833    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal" podStartSLOduration=5.109812828 podStartE2EDuration="5.109812828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:59.985199394 +0000 UTC m=+6.854315955" watchObservedRunningTime="2026-10-30 22:08:00.109812828 +0000 UTC m=+6.978929388"
Oct 30 22:08:00.629459 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:00.629420    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:00.629643 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:00.629606    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:00.629825 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:00.629792    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:00.629943 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:00.629918    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:00.630010 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:00.629997    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:00.630113 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:00.630094    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:00.743825 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:00.743739    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-f4lgs" event={"ID":"653cd0d6-b7a2-446a-b677-406e3e8e56ce","Type":"ContainerStarted","Data":"76e98b3b25c77681350e6d78535c8ce05138efe9a8dab82cf46d5a3471b7f3c7"}
Oct 30 22:08:00.752059 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:00.752018    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal" event={"ID":"2c344a83145e337d085fbb4af4282fc3","Type":"ContainerStarted","Data":"28f376441613fc954d0e14210d221f415eecfb3794179f032e85d72923cf200d"}
Oct 30 22:08:00.869468 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:00.869395    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-f4lgs" podStartSLOduration=2.472177069 podStartE2EDuration="6.869375175s" totalImagesPullingTime="4.397198106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:00.868678742 +0000 UTC m=+7.737795326" watchObservedRunningTime="2026-10-30 22:08:00.869375175 +0000 UTC m=+7.738491734"
Oct 30 22:08:02.477114 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:02.477072    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:02.477595 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:02.477162    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:02.477595 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:02.477311    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:08:02.477595 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:02.477380    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret podName:796116fb-6bb5-4c50-8b4f-9aabe94ea8ce nodeName:}" failed. No retries permitted until 2026-10-30 22:08:10.477361288 +0000 UTC m=+17.346477844 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret") pod "global-pull-secret-syncer-jxjd5" (UID: "796116fb-6bb5-4c50-8b4f-9aabe94ea8ce") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:08:02.477595 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:02.477454    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:08:02.477595 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:02.477487    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs podName:7c315adb-397e-4418-b193-74697cbd6b80 nodeName:}" failed. No retries permitted until 2026-10-30 22:08:10.477476319 +0000 UTC m=+17.346592866 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs") pod "network-metrics-daemon-fgbcv" (UID: "7c315adb-397e-4418-b193-74697cbd6b80") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:08:02.578090 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:02.578020    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7dn7\" (UniqueName: \"kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7\") pod \"network-check-target-s5qdj\" (UID: \"a414991b-4e9e-42e7-be00-a2b460ea3b2b\") " pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:02.578262 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:02.578243    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:08:02.578327 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:02.578269    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:08:02.578327 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:02.578283    2568 projected.go:196] Error preparing data for projected volume kube-api-access-b7dn7 for pod openshift-network-diagnostics/network-check-target-s5qdj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:08:02.578427 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:02.578352    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7 podName:a414991b-4e9e-42e7-be00-a2b460ea3b2b nodeName:}" failed. No retries permitted until 2026-10-30 22:08:10.578325842 +0000 UTC m=+17.447442380 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7dn7" (UniqueName: "kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7") pod "network-check-target-s5qdj" (UID: "a414991b-4e9e-42e7-be00-a2b460ea3b2b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:08:02.629193 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:02.629144    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:02.629366 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:02.629146    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:02.629366 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:02.629306    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:02.629366 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:02.629160    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:02.629537 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:02.629371    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:02.629537 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:02.629422    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:02.759543 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:02.759510    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-krfnw" event={"ID":"515e0d50-7bf5-476d-a84b-79e5f21f959f","Type":"ContainerStarted","Data":"fabda453df7a40c125fc70db47d621bc6e44485383b36d46c656eac37cf471f8"}
Oct 30 22:08:02.763935 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:02.763832    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j" event={"ID":"0f6f4765-3e5c-441e-a388-8f56a6c180a8","Type":"ContainerStarted","Data":"1d93392123d06c0235c5eca8b3229ac24678f13a67a66bf807b7ba954d3497ca"}
Oct 30 22:08:02.889129 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:02.888486    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal" podStartSLOduration=8.888466198 podStartE2EDuration="8.888466198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:00.995944704 +0000 UTC m=+7.865061279" watchObservedRunningTime="2026-10-30 22:08:02.888466198 +0000 UTC m=+9.757582758"
Oct 30 22:08:04.629761 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:04.628865    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:04.629761 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:04.629014    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:04.629761 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:04.629447    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:04.629761 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:04.629553    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:04.629761 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:04.629623    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:04.629761 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:04.629697    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:04.897434 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:04.897408    2568 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 22:08:04.992121 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:04.991450    2568 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:08:04.992121 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:04.992114    2568 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:08:05.119480 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:05.119430    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-krfnw" podStartSLOduration=4.873514212 podStartE2EDuration="11.119412503s" totalImagesPullingTime="6.245898291s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:02.889062547 +0000 UTC m=+9.758179101" watchObservedRunningTime="2026-10-30 22:08:05.119412503 +0000 UTC m=+11.988529061"
Oct 30 22:08:05.522909 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:05.522638    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:08:05.523329 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:05.523306    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:08:05.562724 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:05.562580    2568 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T22:08:04.897431559Z","UUID":"cf75f3c1-6d04-42a3-a38e-94991bb6b841","Handler":null,"Name":"","Endpoint":""}
Oct 30 22:08:05.564512 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:05.564494    2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 22:08:05.564663 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:05.564525    2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 22:08:05.775954 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:05.775920    2568 generic.go:356] "Generic (PLEG): container finished" podID="66d962d8-e0a3-4c4e-b310-e19aa5360593" containerID="fb160548c91d457ead553fdcf734c3a6ecc04b63b88d48a81334fb946db268f1" exitCode=0
Oct 30 22:08:05.776380 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:05.775976    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pndld" event={"ID":"66d962d8-e0a3-4c4e-b310-e19aa5360593","Type":"ContainerDied","Data":"fb160548c91d457ead553fdcf734c3a6ecc04b63b88d48a81334fb946db268f1"}
Oct 30 22:08:05.779199 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:05.779169    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lr7w8" event={"ID":"62af32b1-6029-4da7-b299-0d3ba5746dc6","Type":"ContainerStarted","Data":"d63e83d88e0dfb6857f54ad0b3d573c89672e1bda172c197485bf03217227491"}
Oct 30 22:08:05.782272 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:05.782238    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cshfp" event={"ID":"f122e9a3-7599-48d2-b5fc-1d373d7cf446","Type":"ContainerStarted","Data":"a7fd88765a8c84da8b04430779726133bc3dfea8ce0b9f7ba17139b8c6f47702"}
Oct 30 22:08:05.784770 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:05.784739    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j" event={"ID":"0f6f4765-3e5c-441e-a388-8f56a6c180a8","Type":"ContainerStarted","Data":"1bc2e5f245d976db3f56e3f02a0b678954368579e19363e1fb6bec04faa229f1"}
Oct 30 22:08:06.037947 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:06.037851    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-cshfp" podStartSLOduration=3.368620182 podStartE2EDuration="12.03783515s" totalImagesPullingTime="8.669214968s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:06.03692349 +0000 UTC m=+12.906040047" watchObservedRunningTime="2026-10-30 22:08:06.03783515 +0000 UTC m=+12.906951709"
Oct 30 22:08:06.628895 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:06.628868    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:06.629035 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:06.628947    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:06.629035 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:06.628982    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:06.629143 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:06.629083    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:06.629217 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:06.629174    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:06.629217 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:06.629177    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:06.790745 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:06.790318    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j" event={"ID":"0f6f4765-3e5c-441e-a388-8f56a6c180a8","Type":"ContainerStarted","Data":"889bd9e066010decc407772f37545c05e8d7ad4d9f9ac03deae155b126f3faaf"}
Oct 30 22:08:06.792817 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:06.792221    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nb5dx" event={"ID":"cf8fa689-675c-4153-a857-07894dba2fe6","Type":"ContainerStarted","Data":"092eb96933cbe5cd1929add8110f029100bb2680afe966b133151f0506a4bea5"}
Oct 30 22:08:06.921230 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:06.921156    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j" podStartSLOduration=2.809540208 podStartE2EDuration="12.921134898s" totalImagesPullingTime="10.11159469s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:06.920066368 +0000 UTC m=+13.789182951" watchObservedRunningTime="2026-10-30 22:08:06.921134898 +0000 UTC m=+13.790251456"
Oct 30 22:08:06.921413 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:06.921331    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-lr7w8" podStartSLOduration=3.948148004 podStartE2EDuration="12.921319894s" totalImagesPullingTime="8.97317189s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:06.165374734 +0000 UTC m=+13.034491290" watchObservedRunningTime="2026-10-30 22:08:06.921319894 +0000 UTC m=+13.790436452"
Oct 30 22:08:07.046417 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:07.046336    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nb5dx" podStartSLOduration=3.716168413 podStartE2EDuration="13.046318766s" totalImagesPullingTime="9.330150353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:07.045890471 +0000 UTC m=+13.915007028" watchObservedRunningTime="2026-10-30 22:08:07.046318766 +0000 UTC m=+13.915435323"
Oct 30 22:08:07.796219 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:07.795697    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-57cq6" event={"ID":"f305ea8d-018d-4c14-a102-de41495bf1a9","Type":"ContainerStarted","Data":"334b17dad68d78de63bb8857e53781ac7ac7845ea611ed110fca3f62746d784e"}
Oct 30 22:08:07.798306 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:07.798279    2568 generic.go:356] "Generic (PLEG): container finished" podID="66d962d8-e0a3-4c4e-b310-e19aa5360593" containerID="2ddeeaa3ac8c3ffddb710ffd10f04e7f1286d4b38b43457a018b2465ccde49b5" exitCode=0
Oct 30 22:08:07.798442 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:07.798374    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pndld" event={"ID":"66d962d8-e0a3-4c4e-b310-e19aa5360593","Type":"ContainerDied","Data":"2ddeeaa3ac8c3ffddb710ffd10f04e7f1286d4b38b43457a018b2465ccde49b5"}
Oct 30 22:08:07.921191 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:07.921141    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-57cq6" podStartSLOduration=2.785214843 podStartE2EDuration="13.921129002s" totalImagesPullingTime="11.135914159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:07.920733101 +0000 UTC m=+14.789849673" watchObservedRunningTime="2026-10-30 22:08:07.921129002 +0000 UTC m=+14.790245588"
Oct 30 22:08:08.629392 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:08.629360    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:08.629579 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:08.629370    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:08.629579 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:08.629498    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:08.629579 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:08.629525    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:08.629715 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:08.629594    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:08.629715 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:08.629689    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:09.805922 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:09.805710    2568 generic.go:356] "Generic (PLEG): container finished" podID="66d962d8-e0a3-4c4e-b310-e19aa5360593" containerID="9075091f9954c443066f449dd7e7768a1d743a4d7aa43f621e8ad783be118a72" exitCode=0
Oct 30 22:08:09.805922 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:09.805803    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pndld" event={"ID":"66d962d8-e0a3-4c4e-b310-e19aa5360593","Type":"ContainerDied","Data":"9075091f9954c443066f449dd7e7768a1d743a4d7aa43f621e8ad783be118a72"}
Oct 30 22:08:10.542967 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:10.542931    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:10.543138 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:10.543009    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:10.543138 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:10.543102    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:08:10.543138 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:10.543114    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:08:10.543296 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:10.543175    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs podName:7c315adb-397e-4418-b193-74697cbd6b80 nodeName:}" failed. No retries permitted until 2026-10-30 22:08:26.543160665 +0000 UTC m=+33.412277196 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs") pod "network-metrics-daemon-fgbcv" (UID: "7c315adb-397e-4418-b193-74697cbd6b80") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:08:10.543296 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:10.543194    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret podName:796116fb-6bb5-4c50-8b4f-9aabe94ea8ce nodeName:}" failed. No retries permitted until 2026-10-30 22:08:26.543184969 +0000 UTC m=+33.412301504 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret") pod "global-pull-secret-syncer-jxjd5" (UID: "796116fb-6bb5-4c50-8b4f-9aabe94ea8ce") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:08:10.628874 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:10.628846    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:10.628874 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:10.628873    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:10.629061 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:10.628889    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:10.629061 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:10.628969    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:10.629061 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:10.629041    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:10.629177 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:10.629102    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:10.644050 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:10.644021    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7dn7\" (UniqueName: \"kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7\") pod \"network-check-target-s5qdj\" (UID: \"a414991b-4e9e-42e7-be00-a2b460ea3b2b\") " pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:10.644205 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:10.644186    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:08:10.644269 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:10.644211    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:08:10.644269 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:10.644223    2568 projected.go:196] Error preparing data for projected volume kube-api-access-b7dn7 for pod openshift-network-diagnostics/network-check-target-s5qdj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:08:10.644388 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:10.644284    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7 podName:a414991b-4e9e-42e7-be00-a2b460ea3b2b nodeName:}" failed. No retries permitted until 2026-10-30 22:08:26.644265174 +0000 UTC m=+33.513381719 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7dn7" (UniqueName: "kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7") pod "network-check-target-s5qdj" (UID: "a414991b-4e9e-42e7-be00-a2b460ea3b2b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:08:12.629234 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:12.629121    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:12.629507 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:12.629163    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:12.629507 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:12.629321    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:12.629507 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:12.629168    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:12.629507 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:12.629377    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:12.629507 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:12.629440    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:12.814872 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:12.814850    2568 generic.go:356] "Generic (PLEG): container finished" podID="66d962d8-e0a3-4c4e-b310-e19aa5360593" containerID="8c2ff06730cfb9601e3ebe2eecf608b302ecdea64f6d44cfcb384484410cf787" exitCode=0
Oct 30 22:08:12.814972 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:12.814931    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pndld" event={"ID":"66d962d8-e0a3-4c4e-b310-e19aa5360593","Type":"ContainerDied","Data":"8c2ff06730cfb9601e3ebe2eecf608b302ecdea64f6d44cfcb384484410cf787"}
Oct 30 22:08:12.817552 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:12.817536    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:08:12.817872 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:12.817854    2568 generic.go:356] "Generic (PLEG): container finished" podID="10c86ffc-dbfb-48d8-a225-edc41c719c6f" containerID="66a0846f42e52cec123734946c799d62e2caad3198483bcaacf0adb17f5f34fe" exitCode=1
Oct 30 22:08:12.817949 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:12.817893    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm" event={"ID":"10c86ffc-dbfb-48d8-a225-edc41c719c6f","Type":"ContainerStarted","Data":"4a0c1fcae34ebee48625b2990298ed7f9d1b9f6070248032204343a79ab1552f"}
Oct 30 22:08:12.818013 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:12.817957    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm" event={"ID":"10c86ffc-dbfb-48d8-a225-edc41c719c6f","Type":"ContainerStarted","Data":"9079c6fd0e7f58626e1948cb816d059147c611e50f54cedb597bc9155a228de8"}
Oct 30 22:08:12.818067 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:12.818010    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm" event={"ID":"10c86ffc-dbfb-48d8-a225-edc41c719c6f","Type":"ContainerStarted","Data":"036699f3599792e1741eefc97499c175bc51fbd74b23cc9b79289e8a42ddbb92"}
Oct 30 22:08:12.818116 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:12.818057    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm" event={"ID":"10c86ffc-dbfb-48d8-a225-edc41c719c6f","Type":"ContainerDied","Data":"66a0846f42e52cec123734946c799d62e2caad3198483bcaacf0adb17f5f34fe"}
Oct 30 22:08:12.818172 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:12.818116    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm" event={"ID":"10c86ffc-dbfb-48d8-a225-edc41c719c6f","Type":"ContainerStarted","Data":"b4fcf73166bbde5df6921b3ad5bf26e1b52cea00ad526dbdb8f79b0209647c75"}
Oct 30 22:08:13.823031 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:13.823000    2568 generic.go:356] "Generic (PLEG): container finished" podID="66d962d8-e0a3-4c4e-b310-e19aa5360593" containerID="8dcabc088a0dfcf078b8e932c6199630865300ffd381b25897b156c7cc176d1a" exitCode=0
Oct 30 22:08:13.823476 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:13.823091    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pndld" event={"ID":"66d962d8-e0a3-4c4e-b310-e19aa5360593","Type":"ContainerDied","Data":"8dcabc088a0dfcf078b8e932c6199630865300ffd381b25897b156c7cc176d1a"}
Oct 30 22:08:13.826004 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:13.825987    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:08:13.826347 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:13.826326    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm" event={"ID":"10c86ffc-dbfb-48d8-a225-edc41c719c6f","Type":"ContainerStarted","Data":"69675d1f6c806158c8825efc5b9ad2f182f1e841baf50a2f62344ced2c3ad6d9"}
Oct 30 22:08:14.628856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:14.628823    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:14.628856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:14.628846    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:14.629065 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:14.628946    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:14.629065 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:14.628952    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:14.629065 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:14.629036    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:14.629153 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:14.629103    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:14.832062 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:14.832033    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pndld" event={"ID":"66d962d8-e0a3-4c4e-b310-e19aa5360593","Type":"ContainerStarted","Data":"1f8cf1f7cdce2270eb0b87cf7c14f4e73d6852e3faeea70d9cccf5da06fcc875"}
Oct 30 22:08:14.964031 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:14.963988    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pndld" podStartSLOduration=7.787306401 podStartE2EDuration="20.96397604s" totalImagesPullingTime="13.176669639s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:14.963866173 +0000 UTC m=+21.832982707" watchObservedRunningTime="2026-10-30 22:08:14.96397604 +0000 UTC m=+21.833092592"
Oct 30 22:08:15.836558 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:15.836531    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:08:15.836959 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:15.836942    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm" event={"ID":"10c86ffc-dbfb-48d8-a225-edc41c719c6f","Type":"ContainerStarted","Data":"ee2e9840b78983d2c299264d841d98658ef3b7d4f280140773b9141c5213a550"}
Oct 30 22:08:16.629057 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:16.629025    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:16.629057 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:16.629040    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:16.629345 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:16.629115    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:16.629345 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:16.629177    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:16.629345 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:16.629247    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:16.629345 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:16.629333    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:17.846254 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:17.846077    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:08:17.846892 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:17.846661    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm" event={"ID":"10c86ffc-dbfb-48d8-a225-edc41c719c6f","Type":"ContainerStarted","Data":"34d0d6ede5709b0f118af9c35b1a8767079918790a4b02d2c662b692faccdea2"}
Oct 30 22:08:17.847021 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:17.847003    2568 scope.go:122] "RemoveContainer" containerID="66a0846f42e52cec123734946c799d62e2caad3198483bcaacf0adb17f5f34fe"
Oct 30 22:08:17.847108 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:17.847010    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:08:17.847160 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:17.847153    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:08:17.863367 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:17.863344    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:08:17.863793 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:17.863774    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:08:18.628373 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:18.628339    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:18.628547 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:18.628428    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:18.628547 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:18.628446    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:18.628547 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:18.628533    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:18.628723 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:18.628620    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:18.628723 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:18.628710    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:18.851933 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:18.851907    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:08:18.852320 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:18.852234    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm" event={"ID":"10c86ffc-dbfb-48d8-a225-edc41c719c6f","Type":"ContainerStarted","Data":"f720061b2ba2cb013dcb6b926b241ad5e61345917be4dec9ad2fe829adcb7135"}
Oct 30 22:08:18.852359 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:18.852328    2568 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 22:08:18.983156 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:18.983082    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm" podStartSLOduration=8.594921762 podStartE2EDuration="24.983070723s" totalImagesPullingTime="16.388148961s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:18.982862924 +0000 UTC m=+25.851979488" watchObservedRunningTime="2026-10-30 22:08:18.983070723 +0000 UTC m=+25.852187275"
Oct 30 22:08:19.854466 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:19.854406    2568 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 22:08:19.892597 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:19.892569    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jxjd5"]
Oct 30 22:08:19.892735 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:19.892688    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:19.892797 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:19.892771    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:19.896318 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:19.896291    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fgbcv"]
Oct 30 22:08:19.896437 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:19.896399    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:19.896526 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:19.896505    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:19.896975 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:19.896955    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-s5qdj"]
Oct 30 22:08:19.897051 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:19.897038    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:19.897155 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:19.897136    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:21.629283 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:21.629251    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:21.629740 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:21.629305    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:21.629740 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:21.629404    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:21.629740 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:21.629436    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:21.629740 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:21.629533    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:21.629740 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:21.629583    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:23.630143 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:23.630109    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:23.630615 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:23.630214    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:23.630615 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:23.630257    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:23.630615 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:23.630371    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:23.630615 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:23.630381    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:23.630615 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:23.630497    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:24.101296 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:24.101219    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:08:24.101452 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:24.101438    2568 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 22:08:24.113477 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:24.113315    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:08:25.628078 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:25.628049    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f4lgs_653cd0d6-b7a2-446a-b677-406e3e8e56ce/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:25.628871 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:25.628858    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:25.628921 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:25.628883    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:25.628973 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:25.628954    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:25.629043 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:25.629020    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:25.629075 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:25.629064    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:25.629168 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:25.629151    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:25.962515 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:25.962455    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lr7w8_62af32b1-6029-4da7-b299-0d3ba5746dc6/dns-node-resolver/0.log"
Oct 30 22:08:26.381964 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:26.381884    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f4lgs_653cd0d6-b7a2-446a-b677-406e3e8e56ce/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:26.562138 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:26.562112    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:26.562327 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:26.562176    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:26.562327 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:26.562274    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:08:26.562327 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:26.562303    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:08:26.562460 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:26.562339    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs podName:7c315adb-397e-4418-b193-74697cbd6b80 nodeName:}" failed. No retries permitted until 2026-10-30 22:08:58.56232388 +0000 UTC m=+65.431440436 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs") pod "network-metrics-daemon-fgbcv" (UID: "7c315adb-397e-4418-b193-74697cbd6b80") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:08:26.562460 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:26.562365    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret podName:796116fb-6bb5-4c50-8b4f-9aabe94ea8ce nodeName:}" failed. No retries permitted until 2026-10-30 22:08:58.562346133 +0000 UTC m=+65.431462690 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret") pod "global-pull-secret-syncer-jxjd5" (UID: "796116fb-6bb5-4c50-8b4f-9aabe94ea8ce") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:08:26.662892 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:26.662824    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7dn7\" (UniqueName: \"kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7\") pod \"network-check-target-s5qdj\" (UID: \"a414991b-4e9e-42e7-be00-a2b460ea3b2b\") " pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:26.663230 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:26.662953    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:08:26.663230 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:26.662969    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:08:26.663230 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:26.662980    2568 projected.go:196] Error preparing data for projected volume kube-api-access-b7dn7 for pod openshift-network-diagnostics/network-check-target-s5qdj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:08:26.663230 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:26.663022    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7 podName:a414991b-4e9e-42e7-be00-a2b460ea3b2b nodeName:}" failed. No retries permitted until 2026-10-30 22:08:58.663011761 +0000 UTC m=+65.532128292 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7dn7" (UniqueName: "kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7") pod "network-check-target-s5qdj" (UID: "a414991b-4e9e-42e7-be00-a2b460ea3b2b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:08:27.153080 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:27.153008    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-krfnw_515e0d50-7bf5-476d-a84b-79e5f21f959f/node-ca/0.log"
Oct 30 22:08:27.628435 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:27.628354    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:27.628743 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:27.628356    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:27.628743 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:27.628473    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:27.628743 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:27.628367    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:27.628743 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:27.628724    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:27.628942 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:27.628788    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:29.628786 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:29.628744    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:29.629291 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:29.628744    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:29.629291 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:29.628844    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:29.629291 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:29.628935    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:29.629291 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:29.628745    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:29.629291 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:29.629003    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:31.629183 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:31.629157    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:31.629183 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:31.629175    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:31.629615 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:31.629279    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:31.629615 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:31.629324    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:31.629615 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:31.629358    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:31.629615 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:31.629478    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:33.630555 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:33.630522    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:33.630947 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:33.630600    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:33.630947 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:33.630648    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:33.630947 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:33.630665    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:33.630947 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:33.630745    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:33.630947 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:33.630832    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:35.629238 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:35.629207    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:35.629238 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:35.629236    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:35.629789 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:35.629381    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:35.629789 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:35.629415    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:35.629789 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:35.629491    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:35.629789 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:35.629547    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:37.628842 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:37.628812    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:37.629293 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:37.628819    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:37.629293 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:37.628945    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:37.629293 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:37.628818    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:37.629293 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:37.629002    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:37.629293 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:37.629064    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:39.628306 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:39.628271    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:39.628747 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:39.628382    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:39.628747 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:39.628405    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:39.628747 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:39.628454    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:39.628747 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:39.628527    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:39.628747 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:39.628620    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:41.629072 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:41.629037    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:41.629072 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:41.629049    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:41.629531 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:41.629145    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:41.629531 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:41.629219    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:41.629531 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:41.629256    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:41.629531 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:41.629344    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:43.630137 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:43.630096    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:43.630631 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:43.630173    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:43.630631 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:43.630234    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgbcv" podUID="7c315adb-397e-4418-b193-74697cbd6b80"
Oct 30 22:08:43.630631 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:43.630234    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s5qdj" podUID="a414991b-4e9e-42e7-be00-a2b460ea3b2b"
Oct 30 22:08:43.630631 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:43.630245    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:43.630631 ip-10-0-138-202 kubenswrapper[2568]: E1030 22:08:43.630347    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxjd5" podUID="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce"
Oct 30 22:08:44.867651 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:44.867622    2568 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeReady"
Oct 30 22:08:44.868194 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:44.867752    2568 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 22:08:45.029585 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.029562    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-q4cnn"]
Oct 30 22:08:45.059963 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.059943    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lv7gq"]
Oct 30 22:08:45.060095 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.060077    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:08:45.074038 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.074010    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dmwlw"]
Oct 30 22:08:45.074137 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.074118    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lv7gq"
Oct 30 22:08:45.088262 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.088243    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lv7gq"]
Oct 30 22:08:45.088350 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.088273    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q4cnn"]
Oct 30 22:08:45.088350 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.088289    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dmwlw"]
Oct 30 22:08:45.088350 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.088327    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.113677 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.113632    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 22:08:45.113870 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.113856    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 22:08:45.113870 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.113863    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qzh2l\""
Oct 30 22:08:45.127447 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.127417    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 22:08:45.131180 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.131150    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 22:08:45.131300 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.131288    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 22:08:45.131476 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.131465    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-kpsh7\""
Oct 30 22:08:45.141048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.141007    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 22:08:45.141048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.141020    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nff88\""
Oct 30 22:08:45.141048 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.141020    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 22:08:45.159336 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.159320    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 22:08:45.159803 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.159789    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 22:08:45.193006 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.192987    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-metrics-tls\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:08:45.193103 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.193014    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/211b8165-d716-4097-b371-28ac08591b51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.193103 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.193031    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-config-volume\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:08:45.193103 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.193046    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/211b8165-d716-4097-b371-28ac08591b51-crio-socket\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.193103 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.193071    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfzpz\" (UniqueName: \"kubernetes.io/projected/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-kube-api-access-qfzpz\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:08:45.193245 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.193122    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/211b8165-d716-4097-b371-28ac08591b51-data-volume\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.193245 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.193165    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhvbt\" (UniqueName: \"kubernetes.io/projected/211b8165-d716-4097-b371-28ac08591b51-kube-api-access-qhvbt\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.193245 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.193203    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9891960e-4422-4301-a5c4-404bff9979f9-cert\") pod \"ingress-canary-lv7gq\" (UID: \"9891960e-4422-4301-a5c4-404bff9979f9\") " pod="openshift-ingress-canary/ingress-canary-lv7gq"
Oct 30 22:08:45.193245 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.193222    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gv9f\" (UniqueName: \"kubernetes.io/projected/9891960e-4422-4301-a5c4-404bff9979f9-kube-api-access-5gv9f\") pod \"ingress-canary-lv7gq\" (UID: \"9891960e-4422-4301-a5c4-404bff9979f9\") " pod="openshift-ingress-canary/ingress-canary-lv7gq"
Oct 30 22:08:45.193361 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.193292    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/211b8165-d716-4097-b371-28ac08591b51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.193361 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.193336    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-tmp-dir\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:08:45.294298 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.294262    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-metrics-tls\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:08:45.294415 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.294317    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/211b8165-d716-4097-b371-28ac08591b51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.294415 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.294353    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-config-volume\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:08:45.294415 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.294378    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/211b8165-d716-4097-b371-28ac08591b51-crio-socket\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.294566 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.294512    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfzpz\" (UniqueName: \"kubernetes.io/projected/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-kube-api-access-qfzpz\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:08:45.294566 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.294547    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/211b8165-d716-4097-b371-28ac08591b51-data-volume\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.294687 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.294571    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhvbt\" (UniqueName: \"kubernetes.io/projected/211b8165-d716-4097-b371-28ac08591b51-kube-api-access-qhvbt\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.294687 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.294601    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9891960e-4422-4301-a5c4-404bff9979f9-cert\") pod \"ingress-canary-lv7gq\" (UID: \"9891960e-4422-4301-a5c4-404bff9979f9\") " pod="openshift-ingress-canary/ingress-canary-lv7gq"
Oct 30 22:08:45.294780 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.294747    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gv9f\" (UniqueName: \"kubernetes.io/projected/9891960e-4422-4301-a5c4-404bff9979f9-kube-api-access-5gv9f\") pod \"ingress-canary-lv7gq\" (UID: \"9891960e-4422-4301-a5c4-404bff9979f9\") " pod="openshift-ingress-canary/ingress-canary-lv7gq"
Oct 30 22:08:45.294823 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.294805    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/211b8165-d716-4097-b371-28ac08591b51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.297285 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.297178    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/211b8165-d716-4097-b371-28ac08591b51-data-volume\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.297459 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.297432    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/211b8165-d716-4097-b371-28ac08591b51-crio-socket\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.297708 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.297681    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-tmp-dir\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:08:45.297939 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.297908    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-config-volume\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:08:45.298074 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.297958    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-tmp-dir\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:08:45.298213 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.298189    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/211b8165-d716-4097-b371-28ac08591b51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.299557 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.299525    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-metrics-tls\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:08:45.299661 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.299592    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/211b8165-d716-4097-b371-28ac08591b51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.300051 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.300030    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9891960e-4422-4301-a5c4-404bff9979f9-cert\") pod \"ingress-canary-lv7gq\" (UID: \"9891960e-4422-4301-a5c4-404bff9979f9\") " pod="openshift-ingress-canary/ingress-canary-lv7gq"
Oct 30 22:08:45.357877 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.357850    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfzpz\" (UniqueName: \"kubernetes.io/projected/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-kube-api-access-qfzpz\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:08:45.359716 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.359695    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gv9f\" (UniqueName: \"kubernetes.io/projected/9891960e-4422-4301-a5c4-404bff9979f9-kube-api-access-5gv9f\") pod \"ingress-canary-lv7gq\" (UID: \"9891960e-4422-4301-a5c4-404bff9979f9\") " pod="openshift-ingress-canary/ingress-canary-lv7gq"
Oct 30 22:08:45.359793 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.359740    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhvbt\" (UniqueName: \"kubernetes.io/projected/211b8165-d716-4097-b371-28ac08591b51-kube-api-access-qhvbt\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.394698 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.394653    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:08:45.410832 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.410819    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lv7gq"
Oct 30 22:08:45.447382 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.447365    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:08:45.628613 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.628587    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:45.628850 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.628826    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:45.628920 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.628909    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:45.686352 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.686330    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 22:08:45.686445 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.686381    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 22:08:45.686486 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.686469    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xjxjz\""
Oct 30 22:08:45.686546 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.686527    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 22:08:45.686604 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.686585    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 22:08:45.686683 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.686624    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-f5pfp\""
Oct 30 22:08:45.686732 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.686711    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-c8f2h\""
Oct 30 22:08:45.838688 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.838670    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lv7gq"]
Oct 30 22:08:45.841511 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:08:45.841487    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9891960e_4422_4301_a5c4_404bff9979f9.slice/crio-d9455e225f46284cda0e4a57b908471ea9595613fcab44132976c1b13e833fa6 WatchSource:0}: Error finding container d9455e225f46284cda0e4a57b908471ea9595613fcab44132976c1b13e833fa6: Status 404 returned error can't find the container with id d9455e225f46284cda0e4a57b908471ea9595613fcab44132976c1b13e833fa6
Oct 30 22:08:45.888606 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.888584    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q4cnn"]
Oct 30 22:08:45.892825 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.892794    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dmwlw"]
Oct 30 22:08:45.895253 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:08:45.895235    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod211b8165_d716_4097_b371_28ac08591b51.slice/crio-b689a7577f03db498e71f937ce7caa93f0287304385a56c326c794b784410754 WatchSource:0}: Error finding container b689a7577f03db498e71f937ce7caa93f0287304385a56c326c794b784410754: Status 404 returned error can't find the container with id b689a7577f03db498e71f937ce7caa93f0287304385a56c326c794b784410754
Oct 30 22:08:45.905657 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.905620    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dmwlw" event={"ID":"211b8165-d716-4097-b371-28ac08591b51","Type":"ContainerStarted","Data":"b689a7577f03db498e71f937ce7caa93f0287304385a56c326c794b784410754"}
Oct 30 22:08:45.906808 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.906783    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lv7gq" event={"ID":"9891960e-4422-4301-a5c4-404bff9979f9","Type":"ContainerStarted","Data":"d9455e225f46284cda0e4a57b908471ea9595613fcab44132976c1b13e833fa6"}
Oct 30 22:08:45.907937 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:45.907915    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q4cnn" event={"ID":"8ec5f65f-9ac2-4ad3-812f-781cfd89d855","Type":"ContainerStarted","Data":"2240a51eada2ade5538ed38ac3c92db4477f9c12b77115c515b52657a5fb91c3"}
Oct 30 22:08:46.913271 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:46.913233    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dmwlw" event={"ID":"211b8165-d716-4097-b371-28ac08591b51","Type":"ContainerStarted","Data":"308da27f116dedc1d3141a92c8c74adf848e3706716ee183bfc1836fd8de70bb"}
Oct 30 22:08:47.423350 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.423313    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"]
Oct 30 22:08:47.425472 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.425445    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:08:47.512858 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.512828    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7eac1f43-42ac-4e65-be08-ca44155cee2d-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:08:47.513017 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.512922    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh6cl\" (UniqueName: \"kubernetes.io/projected/7eac1f43-42ac-4e65-be08-ca44155cee2d-kube-api-access-xh6cl\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:08:47.513017 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.512981    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7eac1f43-42ac-4e65-be08-ca44155cee2d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:08:47.513095 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.513036    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eac1f43-42ac-4e65-be08-ca44155cee2d-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:08:47.529035 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.529002    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-lsrrd\""
Oct 30 22:08:47.529194 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.529073    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 22:08:47.529194 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.529105    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 22:08:47.529194 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.529161    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 22:08:47.529194 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.529163    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 22:08:47.569875 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.569833    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 22:08:47.580001 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.579970    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"]
Oct 30 22:08:47.613580 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.613544    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eac1f43-42ac-4e65-be08-ca44155cee2d-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:08:47.613749 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.613662    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7eac1f43-42ac-4e65-be08-ca44155cee2d-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:08:47.613749 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.613702    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh6cl\" (UniqueName: \"kubernetes.io/projected/7eac1f43-42ac-4e65-be08-ca44155cee2d-kube-api-access-xh6cl\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:08:47.613749 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.613735    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7eac1f43-42ac-4e65-be08-ca44155cee2d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:08:47.614402 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.614355    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eac1f43-42ac-4e65-be08-ca44155cee2d-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:08:47.616869 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.616844    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7eac1f43-42ac-4e65-be08-ca44155cee2d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:08:47.617077 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.617030    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7eac1f43-42ac-4e65-be08-ca44155cee2d-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:08:47.677428 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.677338    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh6cl\" (UniqueName: \"kubernetes.io/projected/7eac1f43-42ac-4e65-be08-ca44155cee2d-kube-api-access-xh6cl\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:08:47.739246 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.739219    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:08:47.920937 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:47.920899    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dmwlw" event={"ID":"211b8165-d716-4097-b371-28ac08591b51","Type":"ContainerStarted","Data":"14f0debe073ed352a11c5f27e0f8354985ec011ee57e508c5ff53f1315750b91"}
Oct 30 22:08:48.124772 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:48.124741    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"]
Oct 30 22:08:48.130865 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:08:48.130835    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eac1f43_42ac_4e65_be08_ca44155cee2d.slice/crio-1906a2e4ecb2c11646ebb3d479ba3fa31f8770eb89b919cc6a2f75583f580a1f WatchSource:0}: Error finding container 1906a2e4ecb2c11646ebb3d479ba3fa31f8770eb89b919cc6a2f75583f580a1f: Status 404 returned error can't find the container with id 1906a2e4ecb2c11646ebb3d479ba3fa31f8770eb89b919cc6a2f75583f580a1f
Oct 30 22:08:48.926431 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:48.926385    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q4cnn" event={"ID":"8ec5f65f-9ac2-4ad3-812f-781cfd89d855","Type":"ContainerStarted","Data":"837e06bd2e81955b4c806db9a7300ebe992679c2ba0b4260b503292acbc18956"}
Oct 30 22:08:48.926913 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:48.926461    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q4cnn" event={"ID":"8ec5f65f-9ac2-4ad3-812f-781cfd89d855","Type":"ContainerStarted","Data":"55b53ce2d9a3306dd78856dfa53a13fdfbe52a786892de3cdbad5187d2f31fef"}
Oct 30 22:08:48.927163 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:48.927142    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:08:48.928787 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:48.928750    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw" event={"ID":"7eac1f43-42ac-4e65-be08-ca44155cee2d","Type":"ContainerStarted","Data":"1906a2e4ecb2c11646ebb3d479ba3fa31f8770eb89b919cc6a2f75583f580a1f"}
Oct 30 22:08:49.045881 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:49.045821    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-q4cnn" podStartSLOduration=2.7974480550000003 podStartE2EDuration="5.045803393s" totalImagesPullingTime="2.248355338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:49.04543294 +0000 UTC m=+55.914549512" watchObservedRunningTime="2026-10-30 22:08:49.045803393 +0000 UTC m=+55.914919947"
Oct 30 22:08:49.933392 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:49.933357    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw" event={"ID":"7eac1f43-42ac-4e65-be08-ca44155cee2d","Type":"ContainerStarted","Data":"655f6c6c9cb74a8177320c5a2416d2e616e19ef601b0345c5e8d471f4e9fe799"}
Oct 30 22:08:49.933792 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:49.933409    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw" event={"ID":"7eac1f43-42ac-4e65-be08-ca44155cee2d","Type":"ContainerStarted","Data":"704d75e3657e13310060de17668fe4bbf18bdb9ff85681c4b91158c9dc5bb711"}
Oct 30 22:08:49.935440 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:49.935410    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dmwlw" event={"ID":"211b8165-d716-4097-b371-28ac08591b51","Type":"ContainerStarted","Data":"4caee21bb6749c174933c321ab84a6715b3b8f280fddbb3945c2b7a9ff900a5e"}
Oct 30 22:08:49.936843 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:49.936824    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lv7gq" event={"ID":"9891960e-4422-4301-a5c4-404bff9979f9","Type":"ContainerStarted","Data":"3ff922fda88a15f4bccbc00632e304bfdba7aa09365902d0f73d7e2b7935110e"}
Oct 30 22:08:50.051295 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:50.051221    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw" podStartSLOduration=1.423782916 podStartE2EDuration="3.051205884s" totalImagesPullingTime="1.627422968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:50.051005202 +0000 UTC m=+56.920121773" watchObservedRunningTime="2026-10-30 22:08:50.051205884 +0000 UTC m=+56.920322437"
Oct 30 22:08:50.169347 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:50.169277    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lv7gq" podStartSLOduration=2.820448347 podStartE2EDuration="6.169262308s" totalImagesPullingTime="3.348813961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:50.168616575 +0000 UTC m=+57.037733127" watchObservedRunningTime="2026-10-30 22:08:50.169262308 +0000 UTC m=+57.038378863"
Oct 30 22:08:50.288463 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:50.288417    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dmwlw" podStartSLOduration=3.33945992 podStartE2EDuration="6.28840267s" totalImagesPullingTime="2.94894275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:08:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:50.287737616 +0000 UTC m=+57.156854189" watchObservedRunningTime="2026-10-30 22:08:50.28840267 +0000 UTC m=+57.157519222"
Oct 30 22:08:52.995494 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:52.995466    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"]
Oct 30 22:08:52.997454 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:52.997438    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:08:53.051037 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.051010    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 22:08:53.053321 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.053304    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8bebbb6-d965-4af2-91ff-09497cdbe81e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:08:53.053381 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.053342    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8bebbb6-d965-4af2-91ff-09497cdbe81e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:08:53.053381 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.053360    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8bebbb6-d965-4af2-91ff-09497cdbe81e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:08:53.053381 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.053377    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6lr5\" (UniqueName: \"kubernetes.io/projected/c8bebbb6-d965-4af2-91ff-09497cdbe81e-kube-api-access-m6lr5\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:08:53.100094 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.100066    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 22:08:53.100203 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.100125    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xclcc\""
Oct 30 22:08:53.101064 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.101049    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"]
Oct 30 22:08:53.150942 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.150921    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hq4cs"]
Oct 30 22:08:53.152703 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.152687    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"]
Oct 30 22:08:53.152836 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.152821    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.153828 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.153808    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8bebbb6-d965-4af2-91ff-09497cdbe81e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:08:53.153934 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.153855    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8bebbb6-d965-4af2-91ff-09497cdbe81e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:08:53.153934 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.153882    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8bebbb6-d965-4af2-91ff-09497cdbe81e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:08:53.153934 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.153909    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6lr5\" (UniqueName: \"kubernetes.io/projected/c8bebbb6-d965-4af2-91ff-09497cdbe81e-kube-api-access-m6lr5\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:08:53.154540 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.154520    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.154775 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.154563    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8bebbb6-d965-4af2-91ff-09497cdbe81e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:08:53.156408 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.156383    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8bebbb6-d965-4af2-91ff-09497cdbe81e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:08:53.156510 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.156495    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8bebbb6-d965-4af2-91ff-09497cdbe81e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:08:53.219658 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.219616    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6lr5\" (UniqueName: \"kubernetes.io/projected/c8bebbb6-d965-4af2-91ff-09497cdbe81e-kube-api-access-m6lr5\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:08:53.223881 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.223857    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"]
Oct 30 22:08:53.255102 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.255056    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-tls\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.255102 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.255084    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-root\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.255261 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.255106    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d74e018-659b-4322-a280-b207fed7271f-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.255261 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.255122    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-wtmp\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.255261 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.255182    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.255410 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.255286    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.255410 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.255337    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgltv\" (UniqueName: \"kubernetes.io/projected/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-kube-api-access-xgltv\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.255410 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.255401    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-textfile\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.255530 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.255457    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.255530 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.255481    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.255530 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.255498    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkhm6\" (UniqueName: \"kubernetes.io/projected/6d74e018-659b-4322-a280-b207fed7271f-kube-api-access-tkhm6\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.255530 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.255515    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6d74e018-659b-4322-a280-b207fed7271f-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.255726 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.255539    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-metrics-client-ca\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.255726 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.255562    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.255726 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.255709    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-sys\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.276942 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.276922    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sjbnz\""
Oct 30 22:08:53.277097 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.277063    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 22:08:53.277180 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.277098    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-xsh7n\""
Oct 30 22:08:53.277235 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.277178    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 22:08:53.277307 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.277293    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 22:08:53.277357 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.277296    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 22:08:53.277425 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.277409    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 22:08:53.277472 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.277461    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 22:08:53.308479 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.308456    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:08:53.357291 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357069    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.357291 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357114    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgltv\" (UniqueName: \"kubernetes.io/projected/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-kube-api-access-xgltv\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.357291 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357160    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-textfile\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.357291 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357194    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.357291 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357221    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.357291 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357247    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkhm6\" (UniqueName: \"kubernetes.io/projected/6d74e018-659b-4322-a280-b207fed7271f-kube-api-access-tkhm6\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.357291 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357273    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6d74e018-659b-4322-a280-b207fed7271f-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.357750 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357302    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-metrics-client-ca\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.357750 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357330    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.357750 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357382    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-sys\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.357750 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357429    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-tls\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.357750 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357457    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-root\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.357750 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357491    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d74e018-659b-4322-a280-b207fed7271f-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.357750 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357521    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-wtmp\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.357750 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357555    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.357750 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357596    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-sys\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.358236 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357807    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-root\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.358236 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.357928    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-wtmp\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.358537 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.358496    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.358622 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.358536    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-textfile\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.358819 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.358795    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.359588 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.359306    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-metrics-client-ca\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.359588 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.359443    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d74e018-659b-4322-a280-b207fed7271f-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.359951 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.359925    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6d74e018-659b-4322-a280-b207fed7271f-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.361180 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.361150    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-tls\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.362270 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.362229    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.364036 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.364011    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.366433 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.366413    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.419803 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.419780    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkhm6\" (UniqueName: \"kubernetes.io/projected/6d74e018-659b-4322-a280-b207fed7271f-kube-api-access-tkhm6\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.423526 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.423509    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgltv\" (UniqueName: \"kubernetes.io/projected/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-kube-api-access-xgltv\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.522667 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.522597    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sjbnz\""
Oct 30 22:08:53.526748 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.526731    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-xsh7n\""
Oct 30 22:08:53.531844 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.531821    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:08:53.536001 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.535986    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:08:53.543240 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:08:53.543216    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7759bd31_01d0_41bf_a1ee_4a9ddb8221d0.slice/crio-245de05057f59faa3156b1eef737ac16096a75da62f5ec56f269a2dadb32d14e WatchSource:0}: Error finding container 245de05057f59faa3156b1eef737ac16096a75da62f5ec56f269a2dadb32d14e: Status 404 returned error can't find the container with id 245de05057f59faa3156b1eef737ac16096a75da62f5ec56f269a2dadb32d14e
Oct 30 22:08:53.735058 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.734929    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"]
Oct 30 22:08:53.737390 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:08:53.737364    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8bebbb6_d965_4af2_91ff_09497cdbe81e.slice/crio-fcadf27badaf98b1118d9dfdc884d052646e6f29b678f6a1a7c14fb9a0ea768f WatchSource:0}: Error finding container fcadf27badaf98b1118d9dfdc884d052646e6f29b678f6a1a7c14fb9a0ea768f: Status 404 returned error can't find the container with id fcadf27badaf98b1118d9dfdc884d052646e6f29b678f6a1a7c14fb9a0ea768f
Oct 30 22:08:53.919613 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.919592    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"]
Oct 30 22:08:53.921915 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:08:53.921890    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d74e018_659b_4322_a280_b207fed7271f.slice/crio-8d100abf55cb3fe20a2a2b5801b327a2a24eed9f97058ccb937ec20759f0c711 WatchSource:0}: Error finding container 8d100abf55cb3fe20a2a2b5801b327a2a24eed9f97058ccb937ec20759f0c711: Status 404 returned error can't find the container with id 8d100abf55cb3fe20a2a2b5801b327a2a24eed9f97058ccb937ec20759f0c711
Oct 30 22:08:53.952070 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.952045    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h" event={"ID":"c8bebbb6-d965-4af2-91ff-09497cdbe81e","Type":"ContainerStarted","Data":"f4dfd70fbb76b125f633000d3068d6b5cd70329952719bfc528d74469fc4e707"}
Oct 30 22:08:53.952166 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.952094    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h" event={"ID":"c8bebbb6-d965-4af2-91ff-09497cdbe81e","Type":"ContainerStarted","Data":"388c443dad9b3ca8c4e958dcf441d74ad80e7e723b6591422a91bb38df751d56"}
Oct 30 22:08:53.952166 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.952120    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h" event={"ID":"c8bebbb6-d965-4af2-91ff-09497cdbe81e","Type":"ContainerStarted","Data":"fcadf27badaf98b1118d9dfdc884d052646e6f29b678f6a1a7c14fb9a0ea768f"}
Oct 30 22:08:53.953235 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.953211    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hq4cs" event={"ID":"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0","Type":"ContainerStarted","Data":"245de05057f59faa3156b1eef737ac16096a75da62f5ec56f269a2dadb32d14e"}
Oct 30 22:08:53.954369 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:53.954338    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk" event={"ID":"6d74e018-659b-4322-a280-b207fed7271f","Type":"ContainerStarted","Data":"8d100abf55cb3fe20a2a2b5801b327a2a24eed9f97058ccb937ec20759f0c711"}
Oct 30 22:08:55.252876 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.252802    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:08:55.259481 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.259447    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.273141 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.273100    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-volume\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.273252 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.273207    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.273322 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.273239    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.273322 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.273305    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.273427 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.273325    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.273427 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.273387    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.273503 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.273450    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-out\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.273550 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.273529    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.273610 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.273562    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7kdn\" (UniqueName: \"kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-kube-api-access-l7kdn\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.273610 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.273605    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.273735 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.273629    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.273735 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.273687    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-web-config\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.273735 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.273712    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.314599 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.314456    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 22:08:55.314599 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.314519    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 22:08:55.314599 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.314519    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 22:08:55.314599 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.314547    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tc949\""
Oct 30 22:08:55.314599 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.314565    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 22:08:55.314599 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.314568    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 22:08:55.314599 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.314561    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 22:08:55.314599 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.314597    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 22:08:55.375132 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.375098    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.375132 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.375136    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.375358 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.375154    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-out\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.375358 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.375202    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.375358 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.375221    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7kdn\" (UniqueName: \"kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-kube-api-access-l7kdn\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.375358 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.375248    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.375358 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.375264    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.375358 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.375292    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-web-config\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.375358 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.375315    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.375358 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.375347    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-volume\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.375358 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.375364    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.375814 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.375388    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.375814 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.375409    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.375814 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.375517    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.377210 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.377179    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.379827 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.379261    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-out\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.379827 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.379501    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.379827 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.379763    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.380110 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.379977    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.380845 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.380381    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.380845 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.380802    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-volume\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.381175 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.381155    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.383474 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.383451    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-web-config\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.415080 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.415059    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 22:08:55.416856 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.416836    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.460199 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.460162    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:08:55.461867 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.461302    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 22:08:55.469440 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.469415    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7kdn\" (UniqueName: \"kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-kube-api-access-l7kdn\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.470258 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.470208    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.578621 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.578489    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:08:55.738106 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.738077    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"]
Oct 30 22:08:55.741374 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.741351    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.779237 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.779208    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.779362 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.779274    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.779405 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.779384    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxsxd\" (UniqueName: \"kubernetes.io/projected/414ea994-62c2-4250-9c5a-787cddd4b61e-kube-api-access-gxsxd\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.779444 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.779418    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-tls\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.779483 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.779440    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/414ea994-62c2-4250-9c5a-787cddd4b61e-tmp\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.779535 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.779485    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.779535 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.779515    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-grpc-tls\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.779615 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.779546    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.779615 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.779568    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/414ea994-62c2-4250-9c5a-787cddd4b61e-metrics-client-ca\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.795739 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.795703    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 22:08:55.795739 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.795736    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 22:08:55.795987 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.795946    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-92lj7rkq13635\""
Oct 30 22:08:55.795987 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.795956    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 22:08:55.796088 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.795997    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 22:08:55.796088 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.796031    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 22:08:55.796181 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.796086    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2fh8n\""
Oct 30 22:08:55.806774 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.806321    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"]
Oct 30 22:08:55.880686 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.880601    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.880686 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.880668    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.880884 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.880836    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxsxd\" (UniqueName: \"kubernetes.io/projected/414ea994-62c2-4250-9c5a-787cddd4b61e-kube-api-access-gxsxd\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.880884 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.880862    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-tls\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.880992 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.880882    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/414ea994-62c2-4250-9c5a-787cddd4b61e-tmp\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.880992 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.880962    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.881101 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.881001    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-grpc-tls\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.881101 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.881022    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.881101 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.881038    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/414ea994-62c2-4250-9c5a-787cddd4b61e-metrics-client-ca\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.881402 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.881342    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/414ea994-62c2-4250-9c5a-787cddd4b61e-tmp\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.882195 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.882170    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/414ea994-62c2-4250-9c5a-787cddd4b61e-metrics-client-ca\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.883229 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.883202    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.883767 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.883744    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-grpc-tls\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.883852 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.883832    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.883946 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.883930    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-tls\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.884411 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.884391    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.884484 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.884413    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.943627 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.943603    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxsxd\" (UniqueName: \"kubernetes.io/projected/414ea994-62c2-4250-9c5a-787cddd4b61e-kube-api-access-gxsxd\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:55.966631 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.966604    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h" event={"ID":"c8bebbb6-d965-4af2-91ff-09497cdbe81e","Type":"ContainerStarted","Data":"5b5f70d2a159246dccaf36205d1972b18cc9d6d91d450fe4cc973fa7c7eb93de"}
Oct 30 22:08:55.968321 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.968299    2568 generic.go:356] "Generic (PLEG): container finished" podID="7759bd31-01d0-41bf-a1ee-4a9ddb8221d0" containerID="347657abdffcd08e3e5bf9b38783965d7273422e7a7934841c36061bedf5b4f2" exitCode=0
Oct 30 22:08:55.968401 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:55.968339    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hq4cs" event={"ID":"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0","Type":"ContainerDied","Data":"347657abdffcd08e3e5bf9b38783965d7273422e7a7934841c36061bedf5b4f2"}
Oct 30 22:08:56.022333 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:56.022140    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:08:56.030932 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:08:56.028745    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa31988_78b4_4a78_bac0_8dd6ae099bc5.slice/crio-52876bb9d9c8bfdd32770534e2826f94a7495a77322575d113bd25cafd7088b9 WatchSource:0}: Error finding container 52876bb9d9c8bfdd32770534e2826f94a7495a77322575d113bd25cafd7088b9: Status 404 returned error can't find the container with id 52876bb9d9c8bfdd32770534e2826f94a7495a77322575d113bd25cafd7088b9
Oct 30 22:08:56.064938 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:56.061010    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:08:56.090054 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:56.090000    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h" podStartSLOduration=2.543424586 podStartE2EDuration="4.089984478s" totalImagesPullingTime="1.546559892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:56.08795345 +0000 UTC m=+62.957070018" watchObservedRunningTime="2026-10-30 22:08:56.089984478 +0000 UTC m=+62.959101030"
Oct 30 22:08:56.452288 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:56.452180    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"]
Oct 30 22:08:56.454987 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:08:56.454962    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod414ea994_62c2_4250_9c5a_787cddd4b61e.slice/crio-d88aa6de1c3d54915397822d15f9302e62eff73ddfc59a5f5f25350602d75e9c WatchSource:0}: Error finding container d88aa6de1c3d54915397822d15f9302e62eff73ddfc59a5f5f25350602d75e9c: Status 404 returned error can't find the container with id d88aa6de1c3d54915397822d15f9302e62eff73ddfc59a5f5f25350602d75e9c
Oct 30 22:08:56.977430 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:56.977393    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hq4cs" event={"ID":"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0","Type":"ContainerStarted","Data":"a553a3878ee60bd78d365a770e2c43249bb2ea6c714cf3c26ef84bd1638dd432"}
Oct 30 22:08:56.977616 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:56.977457    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hq4cs" event={"ID":"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0","Type":"ContainerStarted","Data":"e821b46612b5799592ef7ad969931d4fa5f672f8c662abb94c879c92a96c1c30"}
Oct 30 22:08:56.979296 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:56.979263    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw" event={"ID":"414ea994-62c2-4250-9c5a-787cddd4b61e","Type":"ContainerStarted","Data":"d88aa6de1c3d54915397822d15f9302e62eff73ddfc59a5f5f25350602d75e9c"}
Oct 30 22:08:56.981695 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:56.981666    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk" event={"ID":"6d74e018-659b-4322-a280-b207fed7271f","Type":"ContainerStarted","Data":"fa3b89b701c222ba923f7bee90e3ec7019855ec025db6bc5646fac539f9f76f8"}
Oct 30 22:08:56.981818 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:56.981730    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk" event={"ID":"6d74e018-659b-4322-a280-b207fed7271f","Type":"ContainerStarted","Data":"e4370240183a472094b4aab667fa0aa245df3bec4f223882a79a5ed4d532f1a3"}
Oct 30 22:08:56.981818 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:56.981763    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk" event={"ID":"6d74e018-659b-4322-a280-b207fed7271f","Type":"ContainerStarted","Data":"b47813ae0f59f11951a902ce24d763f5f14b5cc349ae066ff87735db54c986ca"}
Oct 30 22:08:56.982969 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:56.982943    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa31988-78b4-4a78-bac0-8dd6ae099bc5","Type":"ContainerStarted","Data":"52876bb9d9c8bfdd32770534e2826f94a7495a77322575d113bd25cafd7088b9"}
Oct 30 22:08:57.098434 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:57.098381    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hq4cs" podStartSLOduration=2.6323729460000003 podStartE2EDuration="4.098367428s" totalImagesPullingTime="1.376863087s" totalInitContainerRuntime="89.131395ms" isStatefulPod=true podCreationTimestamp="2026-10-30 22:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:57.097071872 +0000 UTC m=+63.966188430" watchObservedRunningTime="2026-10-30 22:08:57.098367428 +0000 UTC m=+63.967483991"
Oct 30 22:08:57.217877 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:57.217817    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk" podStartSLOduration=2.246075322 podStartE2EDuration="4.217801827s" totalImagesPullingTime="1.971726505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:57.21699463 +0000 UTC m=+64.086111209" watchObservedRunningTime="2026-10-30 22:08:57.217801827 +0000 UTC m=+64.086918399"
Oct 30 22:08:57.790259 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:57.790192    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"]
Oct 30 22:08:57.792280 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:57.792261    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"
Oct 30 22:08:57.846458 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:57.846434    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 22:08:57.846579 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:57.846548    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-9wjrx\""
Oct 30 22:08:57.853003 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:57.852976    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"]
Oct 30 22:08:57.900624 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:57.900589    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-pvxcb\" (UID: \"fc8d2282-fc99-44d1-9d74-00eb93c5a7ec\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"
Oct 30 22:08:57.900624 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:57.900629    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4d2c\" (UniqueName: \"kubernetes.io/projected/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec-kube-api-access-z4d2c\") pod \"monitoring-plugin-77f4f76594-pvxcb\" (UID: \"fc8d2282-fc99-44d1-9d74-00eb93c5a7ec\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"
Oct 30 22:08:57.988969 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:57.988936    2568 generic.go:356] "Generic (PLEG): container finished" podID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerID="8d07db46466466c90462241b0d4bee2de008cd750736078bcec8f5f26c371470" exitCode=0
Oct 30 22:08:57.989119 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:57.989010    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa31988-78b4-4a78-bac0-8dd6ae099bc5","Type":"ContainerDied","Data":"8d07db46466466c90462241b0d4bee2de008cd750736078bcec8f5f26c371470"}
Oct 30 22:08:58.001389 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.001371    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-pvxcb\" (UID: \"fc8d2282-fc99-44d1-9d74-00eb93c5a7ec\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"
Oct 30 22:08:58.001477 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.001399    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4d2c\" (UniqueName: \"kubernetes.io/projected/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec-kube-api-access-z4d2c\") pod \"monitoring-plugin-77f4f76594-pvxcb\" (UID: \"fc8d2282-fc99-44d1-9d74-00eb93c5a7ec\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"
Oct 30 22:08:58.003827 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.003803    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-pvxcb\" (UID: \"fc8d2282-fc99-44d1-9d74-00eb93c5a7ec\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"
Oct 30 22:08:58.010337 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.010319    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7474bd8d9b-c28kq"]
Oct 30 22:08:58.012359 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.012345    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.066482 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.066431    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 22:08:58.066727 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.066623    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 22:08:58.066727 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.066723    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 22:08:58.066860 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.066793    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 22:08:58.066860 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.066832    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-n9qnm\""
Oct 30 22:08:58.066959 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.066920    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bo7a2mdfbp4fu\""
Oct 30 22:08:58.069140 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.069123    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4d2c\" (UniqueName: \"kubernetes.io/projected/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec-kube-api-access-z4d2c\") pod \"monitoring-plugin-77f4f76594-pvxcb\" (UID: \"fc8d2282-fc99-44d1-9d74-00eb93c5a7ec\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"
Oct 30 22:08:58.076015 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.075996    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7474bd8d9b-c28kq"]
Oct 30 22:08:58.102665 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.102627    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/09cc5ce9-e655-4872-a076-43e6686040e8-audit-log\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.102757 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.102682    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-secret-metrics-server-client-certs\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.102757 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.102698    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-secret-metrics-server-tls\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.102952 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.102934    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zlf7\" (UniqueName: \"kubernetes.io/projected/09cc5ce9-e655-4872-a076-43e6686040e8-kube-api-access-4zlf7\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.103019 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.103000    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/09cc5ce9-e655-4872-a076-43e6686040e8-metrics-server-audit-profiles\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.103190 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.103176    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09cc5ce9-e655-4872-a076-43e6686040e8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.103723 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.103489    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-client-ca-bundle\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.103723 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.103660    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"
Oct 30 22:08:58.204477 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.204448    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-client-ca-bundle\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.204596 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.204493    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/09cc5ce9-e655-4872-a076-43e6686040e8-audit-log\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.204596 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.204528    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-secret-metrics-server-client-certs\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.204596 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.204553    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-secret-metrics-server-tls\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.204773 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.204612    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zlf7\" (UniqueName: \"kubernetes.io/projected/09cc5ce9-e655-4872-a076-43e6686040e8-kube-api-access-4zlf7\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.204773 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.204660    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/09cc5ce9-e655-4872-a076-43e6686040e8-metrics-server-audit-profiles\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.204773 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.204704    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09cc5ce9-e655-4872-a076-43e6686040e8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.205317 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.205257    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/09cc5ce9-e655-4872-a076-43e6686040e8-audit-log\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.205445 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.205423    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09cc5ce9-e655-4872-a076-43e6686040e8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.205913 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.205895    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/09cc5ce9-e655-4872-a076-43e6686040e8-metrics-server-audit-profiles\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.207687 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.207665    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-secret-metrics-server-tls\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.207867 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.207837    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-client-ca-bundle\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.207918 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.207837    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-secret-metrics-server-client-certs\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.267676 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.267626    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zlf7\" (UniqueName: \"kubernetes.io/projected/09cc5ce9-e655-4872-a076-43e6686040e8-kube-api-access-4zlf7\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.322709 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.322634    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:08:58.483724 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.483695    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"]
Oct 30 22:08:58.486149 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:08:58.486120    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc8d2282_fc99_44d1_9d74_00eb93c5a7ec.slice/crio-48ff35a9d70b41cb87e6635608260b6f2b6d3ed3f4e72ce008240589df6e23de WatchSource:0}: Error finding container 48ff35a9d70b41cb87e6635608260b6f2b6d3ed3f4e72ce008240589df6e23de: Status 404 returned error can't find the container with id 48ff35a9d70b41cb87e6635608260b6f2b6d3ed3f4e72ce008240589df6e23de
Oct 30 22:08:58.608583 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.608509    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:58.608583 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.608567    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:58.660350 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.660319    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 22:08:58.660350 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.660328    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 22:08:58.671764 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.671737    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:58.671887 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.671746    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:58.699017 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.698997    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7474bd8d9b-c28kq"]
Oct 30 22:08:58.701155 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:08:58.701134    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09cc5ce9_e655_4872_a076_43e6686040e8.slice/crio-770d47fbb4becb4f94c63ed95394ad07136db453fae97b37023d357408eac74d WatchSource:0}: Error finding container 770d47fbb4becb4f94c63ed95394ad07136db453fae97b37023d357408eac74d: Status 404 returned error can't find the container with id 770d47fbb4becb4f94c63ed95394ad07136db453fae97b37023d357408eac74d
Oct 30 22:08:58.709925 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.709901    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7dn7\" (UniqueName: \"kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7\") pod \"network-check-target-s5qdj\" (UID: \"a414991b-4e9e-42e7-be00-a2b460ea3b2b\") " pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:58.761443 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.761414    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 22:08:58.806546 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.806512    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"]
Oct 30 22:08:58.808854 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.808833    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:58.821874 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.821837    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 22:08:58.833574 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.833548    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7dn7\" (UniqueName: \"kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7\") pod \"network-check-target-s5qdj\" (UID: \"a414991b-4e9e-42e7-be00-a2b460ea3b2b\") " pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:58.862805 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.862752    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 22:08:58.863083 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.863065    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 22:08:58.863177 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.863151    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 22:08:58.863177 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.863157    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 22:08:58.863315 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.863302    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 22:08:58.863370 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.863333    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-4ltrt\""
Oct 30 22:08:58.863911 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.863894    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 22:08:58.876042 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.876016    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"]
Oct 30 22:08:58.894813 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.894794    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-c8f2h\""
Oct 30 22:08:58.900776 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.900760    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-f5pfp\""
Oct 30 22:08:58.903874 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.903857    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:08:58.907983 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.907966    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xjxjz\""
Oct 30 22:08:58.910071 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.910056    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:08:58.911426 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.911408    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-secret-telemeter-client\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:58.911513 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.911436    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-metrics-client-ca\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:58.911513 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.911454    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:58.911513 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.911477    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-federate-client-tls\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:58.911513 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.911495    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf4l6\" (UniqueName: \"kubernetes.io/projected/7cec808e-f1bb-4d43-98da-5779799ae953-kube-api-access-sf4l6\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:58.911743 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.911602    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-telemeter-client-tls\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:58.911743 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.911622    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-serving-certs-ca-bundle\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:58.911743 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.911673    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:58.917542 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.917450    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:08:58.998913 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:58.998850    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb" event={"ID":"fc8d2282-fc99-44d1-9d74-00eb93c5a7ec","Type":"ContainerStarted","Data":"48ff35a9d70b41cb87e6635608260b6f2b6d3ed3f4e72ce008240589df6e23de"}
Oct 30 22:08:59.008374 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.008203    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq" event={"ID":"09cc5ce9-e655-4872-a076-43e6686040e8","Type":"ContainerStarted","Data":"770d47fbb4becb4f94c63ed95394ad07136db453fae97b37023d357408eac74d"}
Oct 30 22:08:59.013145 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.012991    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-secret-telemeter-client\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.013145 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.013032    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-metrics-client-ca\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.013145 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.013058    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.013145 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.013084    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-federate-client-tls\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.013145 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.013110    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf4l6\" (UniqueName: \"kubernetes.io/projected/7cec808e-f1bb-4d43-98da-5779799ae953-kube-api-access-sf4l6\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.013145 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.013139    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-telemeter-client-tls\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.013482 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.013164    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-serving-certs-ca-bundle\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.013482 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.013199    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.016160 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.014728    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-serving-certs-ca-bundle\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.016160 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.015307    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-metrics-client-ca\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.016160 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.015991    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.017388 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.016830    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-telemeter-client-tls\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.017388 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.017349    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-secret-telemeter-client\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.018058 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.018037    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-federate-client-tls\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.018741 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.018715    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.078023 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.077990    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf4l6\" (UniqueName: \"kubernetes.io/projected/7cec808e-f1bb-4d43-98da-5779799ae953-kube-api-access-sf4l6\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.125175 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.125108    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:08:59.287328 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.287283    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-s5qdj"]
Oct 30 22:08:59.290813 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:08:59.290788    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda414991b_4e9e_42e7_be00_a2b460ea3b2b.slice/crio-ada0931703a6b3795b0a74e04e016b530dc7a709a9571ab102f46c4626c9d8ac WatchSource:0}: Error finding container ada0931703a6b3795b0a74e04e016b530dc7a709a9571ab102f46c4626c9d8ac: Status 404 returned error can't find the container with id ada0931703a6b3795b0a74e04e016b530dc7a709a9571ab102f46c4626c9d8ac
Oct 30 22:08:59.317466 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.317445    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jxjd5"]
Oct 30 22:08:59.319314 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:08:59.319272    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod796116fb_6bb5_4c50_8b4f_9aabe94ea8ce.slice/crio-8dc5263230917a41c46ec89a61dfb913c0342c027003cabaa30e27ac8c75f607 WatchSource:0}: Error finding container 8dc5263230917a41c46ec89a61dfb913c0342c027003cabaa30e27ac8c75f607: Status 404 returned error can't find the container with id 8dc5263230917a41c46ec89a61dfb913c0342c027003cabaa30e27ac8c75f607
Oct 30 22:08:59.336375 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.336348    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fgbcv"]
Oct 30 22:08:59.339211 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:08:59.339183    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c315adb_397e_4418_b193_74697cbd6b80.slice/crio-8a3f580aa4c1472e7300336eb06559af45470b5b94940393259b0ffaa60da806 WatchSource:0}: Error finding container 8a3f580aa4c1472e7300336eb06559af45470b5b94940393259b0ffaa60da806: Status 404 returned error can't find the container with id 8a3f580aa4c1472e7300336eb06559af45470b5b94940393259b0ffaa60da806
Oct 30 22:08:59.513420 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.513393    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"]
Oct 30 22:08:59.515964 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:08:59.515939    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cec808e_f1bb_4d43_98da_5779799ae953.slice/crio-c95e58300838a2e5c7f3ce23e518cfb1bd869dac276d4c1b23691a4b47e74d6b WatchSource:0}: Error finding container c95e58300838a2e5c7f3ce23e518cfb1bd869dac276d4c1b23691a4b47e74d6b: Status 404 returned error can't find the container with id c95e58300838a2e5c7f3ce23e518cfb1bd869dac276d4c1b23691a4b47e74d6b
Oct 30 22:08:59.946905 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:08:59.946831    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:09:00.015680 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:00.015627    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw" event={"ID":"414ea994-62c2-4250-9c5a-787cddd4b61e","Type":"ContainerStarted","Data":"5402c47909ca19549a3aa3cdb2ed3f95a4be203a5d9cc5e4aa67846e0935cba3"}
Oct 30 22:09:00.015833 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:00.015715    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw" event={"ID":"414ea994-62c2-4250-9c5a-787cddd4b61e","Type":"ContainerStarted","Data":"d3214fbbdad41c6235c14a7acac9cae8b3657ed9c150f142bb12a5d330e92442"}
Oct 30 22:09:00.015833 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:00.015751    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw" event={"ID":"414ea994-62c2-4250-9c5a-787cddd4b61e","Type":"ContainerStarted","Data":"71aa17e6ef9fd8a0a907963c995695b4bd52aefcfdf8ba47e9dd4504766e8b9e"}
Oct 30 22:09:00.017144 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:00.017116    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb" event={"ID":"7cec808e-f1bb-4d43-98da-5779799ae953","Type":"ContainerStarted","Data":"c95e58300838a2e5c7f3ce23e518cfb1bd869dac276d4c1b23691a4b47e74d6b"}
Oct 30 22:09:00.018432 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:00.018408    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fgbcv" event={"ID":"7c315adb-397e-4418-b193-74697cbd6b80","Type":"ContainerStarted","Data":"8a3f580aa4c1472e7300336eb06559af45470b5b94940393259b0ffaa60da806"}
Oct 30 22:09:00.019758 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:00.019712    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jxjd5" event={"ID":"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce","Type":"ContainerStarted","Data":"8dc5263230917a41c46ec89a61dfb913c0342c027003cabaa30e27ac8c75f607"}
Oct 30 22:09:00.021008 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:00.020988    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-s5qdj" event={"ID":"a414991b-4e9e-42e7-be00-a2b460ea3b2b","Type":"ContainerStarted","Data":"ada0931703a6b3795b0a74e04e016b530dc7a709a9571ab102f46c4626c9d8ac"}
Oct 30 22:09:02.037862 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.036634    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb" event={"ID":"fc8d2282-fc99-44d1-9d74-00eb93c5a7ec","Type":"ContainerStarted","Data":"487ac8ffc20c419b1353f13cdb37ad78c457aa5e4c6ec8a7d28c5e55f73cb2c2"}
Oct 30 22:09:02.037862 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.037577    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"
Oct 30 22:09:02.045769 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.044443    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fgbcv" event={"ID":"7c315adb-397e-4418-b193-74697cbd6b80","Type":"ContainerStarted","Data":"155ad321724d1eed8b9da596bed54cc20ad23965e2ca018b17228ce39643a3cd"}
Oct 30 22:09:02.045769 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.044490    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fgbcv" event={"ID":"7c315adb-397e-4418-b193-74697cbd6b80","Type":"ContainerStarted","Data":"eb06649d089a9a7c4be4ea99b59fd57831ee3cf2fc13f24a05e0cf41ba3e7224"}
Oct 30 22:09:02.046502 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.046487    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"
Oct 30 22:09:02.050667 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.050538    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa31988-78b4-4a78-bac0-8dd6ae099bc5","Type":"ContainerStarted","Data":"8bf0e308cab635020adee75a7be6ea9ff4f86b20263cb1c42a517e4d617f85cf"}
Oct 30 22:09:02.050667 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.050588    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa31988-78b4-4a78-bac0-8dd6ae099bc5","Type":"ContainerStarted","Data":"6781b6106e353b46286ed2454fd1c5557241cea3df6e9f1b94097c46f7d47d2c"}
Oct 30 22:09:02.050667 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.050612    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa31988-78b4-4a78-bac0-8dd6ae099bc5","Type":"ContainerStarted","Data":"593361d4247c9f5160d8d4d2d8f88dead266809da0d7ec87bac9ab79dafec66e"}
Oct 30 22:09:02.051355 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.051038    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa31988-78b4-4a78-bac0-8dd6ae099bc5","Type":"ContainerStarted","Data":"4aebbdb74d2638b4f2f7b9ac5c6916bcaf40ac54d094a63c6767f329769111cf"}
Oct 30 22:09:02.051355 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.051109    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa31988-78b4-4a78-bac0-8dd6ae099bc5","Type":"ContainerStarted","Data":"7ea4cac1ffe0addb15b08f364f4bb5ab401843064bb659f3dca4f4f88275903f"}
Oct 30 22:09:02.059390 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.059347    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq" event={"ID":"09cc5ce9-e655-4872-a076-43e6686040e8","Type":"ContainerStarted","Data":"4a3001a9650c933e0768d5539d65119c51d6e9d881952f8a420051ad36c02624"}
Oct 30 22:09:02.065559 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.065195    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw" event={"ID":"414ea994-62c2-4250-9c5a-787cddd4b61e","Type":"ContainerStarted","Data":"5cbaabc8002b9ec62284d15e82e72b9c8c76aff5660e4e75be9fea00e85a1c02"}
Oct 30 22:09:02.065559 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.065239    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw" event={"ID":"414ea994-62c2-4250-9c5a-787cddd4b61e","Type":"ContainerStarted","Data":"2754cc05eda8fa01bec22b30e332e223d8a78de1ebf25b42dba0d3e539c49330"}
Oct 30 22:09:02.065559 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.065262    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw" event={"ID":"414ea994-62c2-4250-9c5a-787cddd4b61e","Type":"ContainerStarted","Data":"2db4271b2f2162ad88faf933674c08e162a58d21334a47c2d0c9d5cbfd1eadcf"}
Oct 30 22:09:02.065559 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.065478    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:02.154332 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.153269    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb" podStartSLOduration=2.45854691 podStartE2EDuration="5.153257011s" totalImagesPullingTime="2.694710101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:02.152943184 +0000 UTC m=+69.022059728" watchObservedRunningTime="2026-10-30 22:09:02.153257011 +0000 UTC m=+69.022373565"
Oct 30 22:09:02.272944 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.272853    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fgbcv" podStartSLOduration=66.035251734 podStartE2EDuration="1m8.272833368s" totalImagesPullingTime="2.237581634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:02.271708591 +0000 UTC m=+69.140825144" watchObservedRunningTime="2026-10-30 22:09:02.272833368 +0000 UTC m=+69.141949921"
Oct 30 22:09:02.514739 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.514684    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq" podStartSLOduration=2.475398504 podStartE2EDuration="5.514664061s" totalImagesPullingTime="3.039265557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:02.513426619 +0000 UTC m=+69.382543201" watchObservedRunningTime="2026-10-30 22:09:02.514664061 +0000 UTC m=+69.383780608"
Oct 30 22:09:02.595654 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.593235    2568 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:09:02.596837 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.596812    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.642691 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.641083    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw" podStartSLOduration=2.781089109 podStartE2EDuration="7.641064252s" totalImagesPullingTime="4.859975143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:02.639007467 +0000 UTC m=+69.508124045" watchObservedRunningTime="2026-10-30 22:09:02.641064252 +0000 UTC m=+69.510180806"
Oct 30 22:09:02.652585 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.652090    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 22:09:02.652585 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.652183    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 22:09:02.652585 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.652258    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 22:09:02.652585 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.652288    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 22:09:02.652585 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.652303    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 22:09:02.652585 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.652310    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 22:09:02.652585 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.652363    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 22:09:02.652585 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.652380    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 22:09:02.652585 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.652421    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 22:09:02.652585 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.652434    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 22:09:02.652585 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.652486    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bq6t45e7eerjf\""
Oct 30 22:09:02.652585 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.652555    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 22:09:02.653264 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.652624    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qknqm\""
Oct 30 22:09:02.653264 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.652675    2568 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 22:09:02.665964 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.665937    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666119 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.665973    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666119 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666007    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666119 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666029    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666119 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666051    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666119 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666078    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh5mn\" (UniqueName: \"kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-kube-api-access-bh5mn\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666119 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666101    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666403 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666129    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666403 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666162    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-web-config\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666403 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666193    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-config\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666403 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666223    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666403 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666255    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666403 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666314    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666403 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666365    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666403 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666391    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666808 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666417    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666808 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666442    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666808 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666468    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666808 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666491    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.666808 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.666514    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-config-out\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.702947 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.702917    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 22:09:02.739867 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.739807    2568 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 22:09:02.742281 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.742188    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:09:02.767243 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767216    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767362 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767270    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767362 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767295    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767362 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767319    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767362 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767347    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767574 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767371    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767574 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767394    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767574 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767418    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-config-out\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767574 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767474    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767574 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767502    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767574 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767534    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767574 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767560    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767946 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767584    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767946 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767614    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh5mn\" (UniqueName: \"kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-kube-api-access-bh5mn\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767946 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767669    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767946 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767714    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767946 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767748    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-web-config\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767946 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767784    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-config\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767946 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767815    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.767946 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.767857    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.770567 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.770545    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.772959 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.771595    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.772959 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.771654    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.772959 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.772427    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.772959 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.772434    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.774526 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.773620    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.774526 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.774226    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-web-config\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.778223 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.775652    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.778223 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.775938    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.778223 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.776920    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.778223 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.777075    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.778223 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.777508    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.778223 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.777812    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.778223 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.778078    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-config\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.778223 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.778188    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-config-out\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.778630 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.778571    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.779345 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.779323    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.780037 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.780005    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.782113 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.782086    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.836906 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.836870    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh5mn\" (UniqueName: \"kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-kube-api-access-bh5mn\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:02.928359 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:02.928262    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:03.078554 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:03.078519    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa31988-78b4-4a78-bac0-8dd6ae099bc5","Type":"ContainerStarted","Data":"2975e488f6a5c9a0dcde3f62fa924cb948ba10db9d0c4c038c4d5fcd1da07037"}
Oct 30 22:09:03.209165 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:03.209111    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.378366107 podStartE2EDuration="8.209092908s" totalImagesPullingTime="4.810774125s" totalInitContainerRuntime="19.952676ms" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:03.207459879 +0000 UTC m=+70.076576431" watchObservedRunningTime="2026-10-30 22:09:03.209092908 +0000 UTC m=+70.078209469"
Oct 30 22:09:03.381772 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:03.381743    2568 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:09:03.385407 ip-10-0-138-202 kubenswrapper[2568]: W1030 22:09:03.385368    2568 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb127d4c4_4878_4467_b936_060ee2bcca83.slice/crio-7c1405656695dd347a4c305286ea14ad41d060a3efd2a1f7ce26df51bb4f26d1 WatchSource:0}: Error finding container 7c1405656695dd347a4c305286ea14ad41d060a3efd2a1f7ce26df51bb4f26d1: Status 404 returned error can't find the container with id 7c1405656695dd347a4c305286ea14ad41d060a3efd2a1f7ce26df51bb4f26d1
Oct 30 22:09:04.085801 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:04.085657    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb" event={"ID":"7cec808e-f1bb-4d43-98da-5779799ae953","Type":"ContainerStarted","Data":"9a39d864ce490b4b6740efa587007bf29a643b58ddd9da6121dade96b6dbd2b0"}
Oct 30 22:09:04.085801 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:04.085738    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb" event={"ID":"7cec808e-f1bb-4d43-98da-5779799ae953","Type":"ContainerStarted","Data":"f1e50f458d8ac26b06afc75139ea2933c8ebf8e032384be9e48e5d6b451baae4"}
Oct 30 22:09:04.085801 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:04.085769    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb" event={"ID":"7cec808e-f1bb-4d43-98da-5779799ae953","Type":"ContainerStarted","Data":"3a7ba6c0ccfafca24e6565148f20fba509695d3290456e71f5ce35d79ce12605"}
Oct 30 22:09:04.087945 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:04.087911    2568 generic.go:356] "Generic (PLEG): container finished" podID="b127d4c4-4878-4467-b936-060ee2bcca83" containerID="595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe" exitCode=0
Oct 30 22:09:04.088100 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:04.088003    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b127d4c4-4878-4467-b936-060ee2bcca83","Type":"ContainerDied","Data":"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe"}
Oct 30 22:09:04.088100 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:04.088077    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b127d4c4-4878-4467-b936-060ee2bcca83","Type":"ContainerStarted","Data":"7c1405656695dd347a4c305286ea14ad41d060a3efd2a1f7ce26df51bb4f26d1"}
Oct 30 22:09:04.210633 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:04.210567    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb" podStartSLOduration=2.574127248 podStartE2EDuration="6.210553275s" totalImagesPullingTime="3.636426027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:04.208308109 +0000 UTC m=+71.077424685" watchObservedRunningTime="2026-10-30 22:09:04.210553275 +0000 UTC m=+71.079669828"
Oct 30 22:09:05.095400 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:05.095315    2568 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-s5qdj" event={"ID":"a414991b-4e9e-42e7-be00-a2b460ea3b2b","Type":"ContainerStarted","Data":"f9757fbe8bd9a3088355efb20dd514b7d5cdbf8dfb15a00b3bcc85d8bf13fff5"}
Oct 30 22:09:05.095842 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:05.095627    2568 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:09:05.213539 ip-10-0-138-202 kubenswrapper[2568]: I1030 22:09:05.211746    2568 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-s5qdj" podStartSLOduration=65.682445512 podStartE2EDuration="1m11.211727565s" totalImagesPullingTime="5.529282053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:05.211141339 +0000 UTC m=+72.080257895" watchObservedRunningTime="2026-10-30 22:09:05.211727565 +0000 UTC m=+72.080844118"
Oct 30 22:09:07.501810 ip-10-0-138-202 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 22:09:07.518198 ip-10-0-138-202 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 22:09:07.518440 ip-10-0-138-202 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 22:09:07.518506 ip-10-0-138-202 systemd[1]: kubelet.service: Consumed 4.256s CPU time.
Oct 30 22:09:07.535301 ip-10-0-138-202 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 22:09:07.626364 ip-10-0-138-202 kubenswrapper[5896]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:09:07.626364 ip-10-0-138-202 kubenswrapper[5896]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 22:09:07.626364 ip-10-0-138-202 kubenswrapper[5896]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:09:07.626364 ip-10-0-138-202 kubenswrapper[5896]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:09:07.629454 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629363    5896 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:09:07.629454 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629450    5896 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:09:07.629454 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629457    5896 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:09:07.629454 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629460    5896 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:09:07.629454 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629464    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629467    5896 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629470    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629474    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629476    5896 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629479    5896 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629481    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629484    5896 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629487    5896 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629489    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629495    5896 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629497    5896 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629500    5896 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629503    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629505    5896 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629508    5896 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629510    5896 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629513    5896 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629515    5896 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629518    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:09:07.629696 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629521    5896 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629524    5896 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629526    5896 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629531    5896 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629534    5896 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629537    5896 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629540    5896 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629543    5896 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629545    5896 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629549    5896 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629551    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629554    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629557    5896 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629560    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629562    5896 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629565    5896 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629570    5896 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629573    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629577    5896 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:09:07.630327 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629579    5896 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629582    5896 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629585    5896 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629587    5896 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629590    5896 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629593    5896 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629595    5896 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629598    5896 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629601    5896 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629603    5896 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629608    5896 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629611    5896 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629613    5896 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629616    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629619    5896 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629622    5896 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629625    5896 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629627    5896 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629630    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629633    5896 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:09:07.630901 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629656    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629661    5896 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629669    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629674    5896 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629679    5896 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629682    5896 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629685    5896 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629689    5896 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629693    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629698    5896 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629703    5896 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629708    5896 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629712    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629715    5896 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629718    5896 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629725    5896 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629728    5896 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629731    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629733    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629738    5896 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:09:07.631531 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629742    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629746    5896 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629750    5896 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629754    5896 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629766    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629771    5896 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629776    5896 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629786    5896 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629791    5896 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629796    5896 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629801    5896 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629806    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629810    5896 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629814    5896 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629817    5896 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629820    5896 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629823    5896 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629826    5896 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629832    5896 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:09:07.632243 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629834    5896 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629837    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629840    5896 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.629843    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631266    5896 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631277    5896 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631282    5896 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631290    5896 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631293    5896 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631296    5896 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631299    5896 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631302    5896 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631304    5896 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631307    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631310    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631312    5896 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631315    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631318    5896 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631321    5896 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631324    5896 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:09:07.633031 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631329    5896 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631331    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631336    5896 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631340    5896 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631344    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631349    5896 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631353    5896 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631356    5896 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631359    5896 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631362    5896 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631366    5896 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631368    5896 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631371    5896 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631376    5896 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631379    5896 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631382    5896 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631385    5896 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631387    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631390    5896 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631393    5896 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:09:07.633699 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631395    5896 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631398    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631400    5896 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631403    5896 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631406    5896 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631411    5896 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631414    5896 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631417    5896 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631420    5896 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631422    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631425    5896 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631428    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631432    5896 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631437    5896 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631440    5896 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631443    5896 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631446    5896 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631449    5896 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631454    5896 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:09:07.634355 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631456    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631460    5896 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631463    5896 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631466    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631471    5896 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631477    5896 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631483    5896 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631487    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631493    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631498    5896 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631505    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631508    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631510    5896 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631514    5896 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631518    5896 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631522    5896 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631526    5896 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631531    5896 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631534    5896 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631539    5896 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:09:07.635041 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631543    5896 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631547    5896 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631551    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631561    5896 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631565    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631568    5896 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631571    5896 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631574    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631577    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631580    5896 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631583    5896 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631585    5896 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631588    5896 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631591    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631593    5896 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631596    5896 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631601    5896 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631604    5896 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631607    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:09:07.635656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631610    5896 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631612    5896 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631615    5896 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631618    5896 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631620    5896 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631623    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631627    5896 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631630    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631699    5896 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631850    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631858    5896 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.631864    5896 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632710    5896 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632719    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632723    5896 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632728    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632732    5896 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632736    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632741    5896 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:09:07.636274 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632745    5896 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632750    5896 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632754    5896 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632758    5896 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632762    5896 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632766    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632771    5896 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632775    5896 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632779    5896 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632784    5896 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632788    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632792    5896 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632796    5896 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632800    5896 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632805    5896 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632809    5896 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632813    5896 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632817    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632821    5896 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632825    5896 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:09:07.636905 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632829    5896 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632834    5896 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632839    5896 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632843    5896 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632847    5896 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632851    5896 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632855    5896 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632859    5896 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632863    5896 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632867    5896 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632871    5896 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632875    5896 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632879    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632884    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632889    5896 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632893    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632907    5896 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632915    5896 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632920    5896 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632925    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:09:07.637512 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632930    5896 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632934    5896 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632938    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632942    5896 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632946    5896 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632951    5896 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632955    5896 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632959    5896 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632963    5896 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632967    5896 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632971    5896 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632976    5896 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632980    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632984    5896 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632988    5896 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632992    5896 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632996    5896 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.632999    5896 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633003    5896 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:09:07.638431 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633007    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633011    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633016    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633020    5896 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633024    5896 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633028    5896 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633032    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633036    5896 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633040    5896 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633044    5896 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633053    5896 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633060    5896 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633064    5896 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633069    5896 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633074    5896 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633079    5896 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633083    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633087    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633091    5896 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:09:07.639015 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633096    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633122    5896 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633128    5896 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633133    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633138    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633142    5896 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633146    5896 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633150    5896 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633154    5896 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633158    5896 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633162    5896 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633167    5896 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633171    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633175    5896 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633179    5896 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633183    5896 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633189    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633194    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633198    5896 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633201    5896 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:09:07.639567 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.633205    5896 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633325    5896 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633338    5896 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633348    5896 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633355    5896 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633361    5896 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633366    5896 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633372    5896 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633379    5896 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633383    5896 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633388    5896 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633394    5896 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633399    5896 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633404    5896 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633409    5896 flags.go:64] FLAG: --cgroup-root=""
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633413    5896 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633418    5896 flags.go:64] FLAG: --client-ca-file=""
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633422    5896 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633427    5896 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633432    5896 flags.go:64] FLAG: --cluster-domain=""
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633436    5896 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633441    5896 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633446    5896 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633451    5896 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 22:09:07.640138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633458    5896 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633462    5896 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633467    5896 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633472    5896 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633476    5896 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633481    5896 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633486    5896 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633490    5896 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633495    5896 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633508    5896 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633514    5896 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633518    5896 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633523    5896 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633527    5896 flags.go:64] FLAG: --enable-server="true"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633532    5896 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633538    5896 flags.go:64] FLAG: --event-burst="100"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633543    5896 flags.go:64] FLAG: --event-qps="50"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633548    5896 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633553    5896 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633560    5896 flags.go:64] FLAG: --eviction-hard=""
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633567    5896 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633571    5896 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633576    5896 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633581    5896 flags.go:64] FLAG: --eviction-soft=""
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633585    5896 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 22:09:07.640857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633590    5896 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633595    5896 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633600    5896 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633604    5896 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633609    5896 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633613    5896 flags.go:64] FLAG: --feature-gates=""
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633619    5896 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633624    5896 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633628    5896 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633633    5896 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633653    5896 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633659    5896 flags.go:64] FLAG: --help="false"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633663    5896 flags.go:64] FLAG: --hostname-override="ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633669    5896 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633674    5896 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633678    5896 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633683    5896 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633689    5896 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633694    5896 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633698    5896 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633702    5896 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633707    5896 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633712    5896 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 22:09:07.641603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633716    5896 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633721    5896 flags.go:64] FLAG: --kube-reserved=""
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633725    5896 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633731    5896 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633736    5896 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633740    5896 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633745    5896 flags.go:64] FLAG: --lock-file=""
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633749    5896 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633754    5896 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633759    5896 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633767    5896 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633772    5896 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633776    5896 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633781    5896 flags.go:64] FLAG: --logging-format="text"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633785    5896 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633790    5896 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633794    5896 flags.go:64] FLAG: --manifest-url=""
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633800    5896 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633807    5896 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633812    5896 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633818    5896 flags.go:64] FLAG: --max-pods="110"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633822    5896 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633827    5896 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633832    5896 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633836    5896 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 22:09:07.642344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633841    5896 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633846    5896 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633852    5896 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633862    5896 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633867    5896 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633872    5896 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633877    5896 flags.go:64] FLAG: --pod-cidr=""
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633881    5896 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633885    5896 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633890    5896 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633896    5896 flags.go:64] FLAG: --port="10250"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633901    5896 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633906    5896 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-060bd5c44b3270e14"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633911    5896 flags.go:64] FLAG: --qos-reserved=""
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633916    5896 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633920    5896 flags.go:64] FLAG: --register-node="true"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633925    5896 flags.go:64] FLAG: --register-with-taints=""
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633931    5896 flags.go:64] FLAG: --registry-burst="10"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633935    5896 flags.go:64] FLAG: --registry-qps="5"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633940    5896 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633945    5896 flags.go:64] FLAG: --reserved-memory=""
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633951    5896 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633955    5896 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633960    5896 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633964    5896 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633969    5896 flags.go:64] FLAG: --runonce="false"
Oct 30 22:09:07.643192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633974    5896 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633979    5896 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633984    5896 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633988    5896 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633993    5896 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.633998    5896 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634003    5896 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634007    5896 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634014    5896 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634018    5896 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634023    5896 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634027    5896 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634032    5896 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634044    5896 flags.go:64] FLAG: --system-cgroups=""
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634048    5896 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634057    5896 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634063    5896 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634067    5896 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634073    5896 flags.go:64] FLAG: --tls-min-version=""
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634077    5896 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634083    5896 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634088    5896 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634093    5896 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634097    5896 flags.go:64] FLAG: --v="2"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634104    5896 flags.go:64] FLAG: --version="false"
Oct 30 22:09:07.644118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634110    5896 flags.go:64] FLAG: --vmodule=""
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634116    5896 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634121    5896 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634324    5896 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634329    5896 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634336    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634340    5896 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634345    5896 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634349    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634353    5896 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634357    5896 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634361    5896 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634365    5896 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634370    5896 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634374    5896 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634380    5896 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634384    5896 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634388    5896 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634393    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634396    5896 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:09:07.644949 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634401    5896 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634405    5896 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634409    5896 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634414    5896 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634418    5896 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634422    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634426    5896 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634430    5896 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634434    5896 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634439    5896 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634443    5896 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634448    5896 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634452    5896 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634456    5896 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634460    5896 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634465    5896 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634468    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634472    5896 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634476    5896 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634481    5896 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:09:07.645814 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634485    5896 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634489    5896 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634493    5896 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634497    5896 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634500    5896 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634504    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634511    5896 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634519    5896 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634525    5896 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634530    5896 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634534    5896 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634538    5896 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634542    5896 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634546    5896 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634550    5896 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634556    5896 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634559    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634563    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634568    5896 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634572    5896 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:09:07.646395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634576    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634580    5896 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634584    5896 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634588    5896 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634592    5896 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634596    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634599    5896 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634604    5896 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634608    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634615    5896 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634621    5896 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634626    5896 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634631    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634635    5896 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634655    5896 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634659    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634664    5896 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634668    5896 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:09:07.647107 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634672    5896 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634678    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634682    5896 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634687    5896 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634691    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634695    5896 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634699    5896 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634703    5896 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634707    5896 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634712    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634716    5896 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634720    5896 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634725    5896 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634728    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634732    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634736    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634740    5896 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634744    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634748    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634752    5896 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:09:07.647558 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634756    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:09:07.648224 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634760    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:09:07.648224 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634765    5896 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:09:07.648224 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634768    5896 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:09:07.648224 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634773    5896 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:09:07.648224 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634776    5896 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:09:07.648224 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634781    5896 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:09:07.648224 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634785    5896 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:09:07.648224 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634789    5896 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:09:07.648224 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634793    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:09:07.648224 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634797    5896 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:09:07.648224 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.634835    5896 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:09:07.648224 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634895    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:09:07.648224 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634901    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:09:07.648224 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634910    5896 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:09:07.648224 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634914    5896 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:09:07.648224 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634919    5896 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634923    5896 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634927    5896 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634931    5896 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634935    5896 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634941    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634946    5896 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634952    5896 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634956    5896 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634961    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634965    5896 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634969    5896 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634973    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634977    5896 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634981    5896 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634985    5896 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634989    5896 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634993    5896 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.634997    5896 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635002    5896 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:09:07.648704 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635006    5896 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635010    5896 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635014    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635018    5896 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635022    5896 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635026    5896 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635030    5896 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635034    5896 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635038    5896 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635043    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635049    5896 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635053    5896 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635058    5896 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635062    5896 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635066    5896 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635070    5896 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635075    5896 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635081    5896 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635085    5896 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:09:07.649395 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635089    5896 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635093    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635097    5896 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635101    5896 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635105    5896 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635111    5896 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635114    5896 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635119    5896 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635123    5896 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635127    5896 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635131    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635137    5896 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635144    5896 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635149    5896 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635154    5896 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635158    5896 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635162    5896 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635166    5896 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635170    5896 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635174    5896 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:09:07.650046 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635179    5896 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635183    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635187    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635192    5896 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635196    5896 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635201    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635205    5896 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635210    5896 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635214    5896 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635218    5896 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635223    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635227    5896 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635231    5896 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635235    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635241    5896 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635247    5896 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635251    5896 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635256    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:09:07.650711 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635261    5896 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635266    5896 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635270    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635275    5896 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635280    5896 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635284    5896 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635288    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635292    5896 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635296    5896 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635300    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635304    5896 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635308    5896 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635312    5896 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635316    5896 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635320    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635324    5896 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635328    5896 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635333    5896 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635337    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635341    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:09:07.651264 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635346    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635350    5896 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635354    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635358    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.635364    5896 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.639845    5896 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.639864    5896 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.639895    5896 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.639901    5896 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.639957    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.639962    5896 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.639965    5896 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.639968    5896 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.639971    5896 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.639974    5896 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.639977    5896 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.639980    5896 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.639982    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.639985    5896 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:09:07.651962 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.639987    5896 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.639990    5896 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.639993    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.639995    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.639998    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640001    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640003    5896 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640006    5896 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640008    5896 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640011    5896 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640014    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640017    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640019    5896 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640022    5896 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640024    5896 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640028    5896 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640032    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640035    5896 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640038    5896 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640041    5896 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:09:07.652656 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640044    5896 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640049    5896 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640053    5896 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640057    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640061    5896 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640065    5896 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640070    5896 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640074    5896 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640079    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640083    5896 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640089    5896 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640093    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640097    5896 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640101    5896 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640105    5896 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640108    5896 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640110    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640113    5896 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640116    5896 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640118    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:09:07.653465 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640121    5896 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640124    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640126    5896 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640129    5896 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640131    5896 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640134    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640136    5896 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640139    5896 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640142    5896 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640144    5896 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640147    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640149    5896 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640153    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640156    5896 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640159    5896 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640161    5896 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640165    5896 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640167    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640170    5896 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640173    5896 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:09:07.654298 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640175    5896 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640178    5896 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640180    5896 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640183    5896 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640185    5896 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640188    5896 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640190    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640192    5896 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640195    5896 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640198    5896 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640200    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640202    5896 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640205    5896 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640208    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640210    5896 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640213    5896 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640215    5896 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640218    5896 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640220    5896 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640223    5896 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:09:07.655134 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640225    5896 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:09:07.656124 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640228    5896 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:09:07.656124 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640231    5896 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:09:07.656124 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640233    5896 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:09:07.656124 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640237    5896 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:09:07.656124 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640241    5896 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:09:07.656124 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640244    5896 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:09:07.656124 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640247    5896 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:09:07.656124 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640249    5896 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:09:07.656124 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640252    5896 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:09:07.656124 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640254    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:09:07.656124 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640257    5896 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:09:07.656124 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640259    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:09:07.656124 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640262    5896 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:09:07.656124 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640264    5896 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:09:07.656124 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640267    5896 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:09:07.656778 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.640295    5896 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:09:07.656778 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640437    5896 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:09:07.656778 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640442    5896 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:09:07.656778 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640445    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:09:07.656778 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640448    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:09:07.656778 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640450    5896 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:09:07.656778 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640453    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:09:07.656778 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640457    5896 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:09:07.656778 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640461    5896 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:09:07.656778 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640463    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:09:07.656778 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640466    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:09:07.656778 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640468    5896 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:09:07.656778 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640471    5896 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:09:07.656778 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640473    5896 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:09:07.656778 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640476    5896 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640480    5896 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640483    5896 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640487    5896 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640490    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640493    5896 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640496    5896 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640499    5896 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640503    5896 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640506    5896 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640508    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640511    5896 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640513    5896 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640516    5896 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640518    5896 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640521    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640524    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640526    5896 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640529    5896 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:09:07.657522 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640531    5896 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640534    5896 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640536    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640540    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640544    5896 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640548    5896 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640553    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640558    5896 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640562    5896 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640567    5896 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640571    5896 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640575    5896 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640579    5896 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640583    5896 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640587    5896 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640592    5896 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640597    5896 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640600    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640620    5896 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640624    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:09:07.658393 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640627    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640630    5896 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640634    5896 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640653    5896 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640658    5896 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640661    5896 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640665    5896 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640668    5896 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640671    5896 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640674    5896 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640676    5896 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640678    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640681    5896 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640683    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640686    5896 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640688    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640691    5896 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640693    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640695    5896 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:09:07.659029 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640698    5896 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640701    5896 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640703    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640706    5896 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640708    5896 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640710    5896 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640713    5896 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640715    5896 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640718    5896 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640721    5896 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640724    5896 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640726    5896 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640729    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640731    5896 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640733    5896 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640736    5896 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640739    5896 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640742    5896 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640744    5896 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640747    5896 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:09:07.659676 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640749    5896 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:09:07.660211 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640751    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:09:07.660211 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640754    5896 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:09:07.660211 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640756    5896 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:09:07.660211 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640759    5896 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:09:07.660211 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640761    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:09:07.660211 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640764    5896 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:09:07.660211 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640766    5896 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:09:07.660211 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640769    5896 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:09:07.660211 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640771    5896 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:09:07.660211 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640774    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:09:07.660211 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640776    5896 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:09:07.660211 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640780    5896 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:09:07.660211 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640782    5896 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:09:07.660211 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640785    5896 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:09:07.660211 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.640813    5896 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:09:07.660211 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640861    5896 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640868    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640872    5896 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640877    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640880    5896 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640883    5896 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640885    5896 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640887    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640891    5896 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640895    5896 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640899    5896 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640903    5896 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640907    5896 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640910    5896 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640912    5896 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640915    5896 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640917    5896 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640920    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640922    5896 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640925    5896 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:09:07.660792 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640930    5896 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640933    5896 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640935    5896 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640938    5896 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640940    5896 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640943    5896 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640945    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640949    5896 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640953    5896 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640957    5896 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640960    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640963    5896 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640966    5896 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640968    5896 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640971    5896 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640973    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640976    5896 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640978    5896 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640981    5896 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640983    5896 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:09:07.661510 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640986    5896 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640988    5896 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640991    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640993    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640996    5896 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.640998    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641003    5896 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641007    5896 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641010    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641013    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641015    5896 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641018    5896 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641027    5896 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641031    5896 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641038    5896 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641044    5896 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641049    5896 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641054    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641058    5896 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:09:07.662180 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641062    5896 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641067    5896 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641071    5896 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641075    5896 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641079    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641082    5896 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641085    5896 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641088    5896 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641090    5896 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641093    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641095    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641098    5896 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641101    5896 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641105    5896 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641110    5896 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641114    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641117    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641119    5896 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641123    5896 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:09:07.662765 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641126    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641129    5896 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641131    5896 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641134    5896 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641137    5896 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641141    5896 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641144    5896 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641147    5896 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641150    5896 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641152    5896 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641155    5896 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641158    5896 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641160    5896 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641163    5896 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641165    5896 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641167    5896 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641170    5896 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641172    5896 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641175    5896 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641178    5896 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:09:07.663440 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641180    5896 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:09:07.664078 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641183    5896 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:09:07.664078 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641187    5896 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:09:07.664078 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641189    5896 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:09:07.664078 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641192    5896 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:09:07.664078 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641195    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:09:07.664078 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641199    5896 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:09:07.664078 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:09:07.641203    5896 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:09:07.664078 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.641272    5896 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 22:09:07.664078 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.642536    5896 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 22:09:07.664078 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.642592    5896 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 22:09:07.664078 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.642828    5896 server.go:1014] "Starting client certificate rotation"
Oct 30 22:09:07.664078 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.642968    5896 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 22:09:07.664078 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.643029    5896 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 22:02:54 +0000 UTC" deadline="2028-04-18 12:05:57.841089606 +0000 UTC"
Oct 30 22:09:07.664078 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.643067    5896 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12853h56m50.198026531s"
Oct 30 22:09:07.664078 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.643406    5896 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 22:09:07.664626 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.644724    5896 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 22:09:07.664626 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.646696    5896 log.go:25] "Validated CRI v1 runtime API"
Oct 30 22:09:07.664626 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.648896    5896 log.go:25] "Validated CRI v1 image API"
Oct 30 22:09:07.664626 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.649733    5896 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 22:09:07.664626 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.652301    5896 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7f371b9b-78c0-479a-a134-4f741cdd6c33:/dev/nvme0n1p4 8b6e8eaf-c32f-46f2-a685-692c172b452b:/dev/nvme0n1p3]
Oct 30 22:09:07.664908 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.652316    5896 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1906a2e4ecb2c11646ebb3d479ba3fa31f8770eb89b919cc6a2f75583f580a1f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1906a2e4ecb2c11646ebb3d479ba3fa31f8770eb89b919cc6a2f75583f580a1f/userdata/shm major:0 minor:231 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2240a51eada2ade5538ed38ac3c92db4477f9c12b77115c515b52657a5fb91c3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2240a51eada2ade5538ed38ac3c92db4477f9c12b77115c515b52657a5fb91c3/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/245de05057f59faa3156b1eef737ac16096a75da62f5ec56f269a2dadb32d14e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/245de05057f59faa3156b1eef737ac16096a75da62f5ec56f269a2dadb32d14e/userdata/shm major:0 minor:279 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3c8636df0f9c440a3e0a695864d0ae42f268c4f81bbec6c28f2d73a30e375eb5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3c8636df0f9c440a3e0a695864d0ae42f268c4f81bbec6c28f2d73a30e375eb5/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/43a9d1f849e1d1cac670ea77815734593a4bc933243c71535cb9504330a0ebc8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/43a9d1f849e1d1cac670ea77815734593a4bc933243c71535cb9504330a0ebc8/userdata/shm major:0 minor:77 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/48ff35a9d70b41cb87e6635608260b6f2b6d3ed3f4e72ce008240589df6e23de/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/48ff35a9d70b41cb87e6635608260b6f2b6d3ed3f4e72ce008240589df6e23de/userdata/shm major:0 minor:358 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4a1d5c898a7bc9ead0f87d7e30de621dba5ea85c36aa89a107ba490375462c36/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4a1d5c898a7bc9ead0f87d7e30de621dba5ea85c36aa89a107ba490375462c36/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/52876bb9d9c8bfdd32770534e2826f94a7495a77322575d113bd25cafd7088b9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/52876bb9d9c8bfdd32770534e2826f94a7495a77322575d113bd25cafd7088b9/userdata/shm major:0 minor:315 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/770d47fbb4becb4f94c63ed95394ad07136db453fae97b37023d357408eac74d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/770d47fbb4becb4f94c63ed95394ad07136db453fae97b37023d357408eac74d/userdata/shm major:0 minor:364 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7c1405656695dd347a4c305286ea14ad41d060a3efd2a1f7ce26df51bb4f26d1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7c1405656695dd347a4c305286ea14ad41d060a3efd2a1f7ce26df51bb4f26d1/userdata/shm major:0 minor:491 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/805374b844f5cde99fd270dd7a7fc0fe4ad77396cdc9175d9b6bb73195eeffb3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/805374b844f5cde99fd270dd7a7fc0fe4ad77396cdc9175d9b6bb73195eeffb3/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/88936199bf332ffa78567925b302310cff7b32634bfa61ef0ce08d322819056b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/88936199bf332ffa78567925b302310cff7b32634bfa61ef0ce08d322819056b/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8a3f580aa4c1472e7300336eb06559af45470b5b94940393259b0ffaa60da806/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8a3f580aa4c1472e7300336eb06559af45470b5b94940393259b0ffaa60da806/userdata/shm major:0 minor:377 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8d100abf55cb3fe20a2a2b5801b327a2a24eed9f97058ccb937ec20759f0c711/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8d100abf55cb3fe20a2a2b5801b327a2a24eed9f97058ccb937ec20759f0c711/userdata/shm major:0 minor:283 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8dc5263230917a41c46ec89a61dfb913c0342c027003cabaa30e27ac8c75f607/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8dc5263230917a41c46ec89a61dfb913c0342c027003cabaa30e27ac8c75f607/userdata/shm major:0 minor:375 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/960b2d65f6ce6ab539de3b3e43c196062928b3c35558d96550ae8ba45854fc6a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/960b2d65f6ce6ab539de3b3e43c196062928b3c35558d96550ae8ba45854fc6a/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ada0931703a6b3795b0a74e04e016b530dc7a709a9571ab102f46c4626c9d8ac/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ada0931703a6b3795b0a74e04e016b530dc7a709a9571ab102f46c4626c9d8ac/userdata/shm major:0 minor:373 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b689a7577f03db498e71f937ce7caa93f0287304385a56c326c794b784410754/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b689a7577f03db498e71f937ce7caa93f0287304385a56c326c794b784410754/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c63c3353e20169010c60f5b5a591841e10c90a80d30f9c47821c7d32c763fed9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c63c3353e20169010c60f5b5a591841e10c90a80d30f9c47821c7d32c763fed9/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c95e58300838a2e5c7f3ce23e518cfb1bd869dac276d4c1b23691a4b47e74d6b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c95e58300838a2e5c7f3ce23e518cfb1bd869dac276d4c1b23691a4b47e74d6b/userdata/shm major:0 minor:384 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d88aa6de1c3d54915397822d15f9302e62eff73ddfc59a5f5f25350602d75e9c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d88aa6de1c3d54915397822d15f9302e62eff73ddfc59a5f5f25350602d75e9c/userdata/shm major:0 minor:354 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d9455e225f46284cda0e4a57b908471ea9595613fcab44132976c1b13e833fa6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d9455e225f46284cda0e4a57b908471ea9595613fcab44132976c1b13e833fa6/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eb7a959b006264436cfb665fe0dc8c1d95f5708fe651d147affac90525cdb86c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eb7a959b006264436cfb665fe0dc8c1d95f5708fe651d147affac90525cdb86c/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fc46b1d3dc62cfa9bde50dc66e27f1d33f92338594cd4d548c722369ac5937b2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fc46b1d3dc62cfa9bde50dc66e27f1d33f92338594cd4d548c722369ac5937b2/userdata/shm major:0 minor:62 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fc69fe88a9799c1453ea823b86041b89808fb2d7690c24d6575b31ce67652821/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fc69fe88a9799c1453ea823b86041b89808fb2d7690c24d6575b31ce67652821/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fcadf27badaf98b1118d9dfdc884d052646e6f29b678f6a1a7c14fb9a0ea768f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fcadf27badaf98b1118d9dfdc884d052646e6f29b678f6a1a7c14fb9a0ea768f/userdata/shm major:0 minor:271 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/09cc5ce9-e655-4872-a076-43e6686040e8/volumes/kubernetes.io~projected/kube-api-access-4zlf7:{mountpoint:/var/lib/kubelet/pods/09cc5ce9-e655-4872-a076-43e6686040e8/volumes/kubernetes.io~projected/kube-api-access-4zlf7 major:0 minor:363 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/09cc5ce9-e655-4872-a076-43e6686040e8/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/09cc5ce9-e655-4872-a076-43e6686040e8/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:361 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/09cc5ce9-e655-4872-a076-43e6686040e8/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/09cc5ce9-e655-4872-a076-43e6686040e8/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:362 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/09cc5ce9-e655-4872-a076-43e6686040e8/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/09cc5ce9-e655-4872-a076-43e6686040e8/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:360 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:304 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volume-subpaths/web-config/alertmanager/9 major:0 minor:307 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volume-subpaths/web-config/config-reloader/6 major:0 minor:307 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:307 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~empty-dir/config-out major:0 minor:301 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~projected/kube-api-access-l7kdn:{mountpoint:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~projected/kube-api-access-l7kdn major:0 minor:310 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~projected/tls-assets major:0 minor:314 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:304 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/config-volume major:0 minor:305 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:302 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:303 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:306 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:300 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/web-config major:0 minor:307 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0f6f4765-3e5c-441e-a388-8f56a6c180a8/volumes/kubernetes.io~projected/kube-api-access-xxwgg:{mountpoint:/var/lib/kubelet/pods/0f6f4765-3e5c-441e-a388-8f56a6c180a8/volumes/kubernetes.io~projected/kube-api-access-xxwgg major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/10c86ffc-dbfb-48d8-a225-edc41c719c6f/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/10c86ffc-dbfb-48d8-a225-edc41c719c6f/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/10c86ffc-dbfb-48d8-a225-edc41c719c6f/volumes/kubernetes.io~projected/kube-api-access-89c5l:{mountpoint:/var/lib/kubelet/pods/10c86ffc-dbfb-48d8-a225-edc41c719c6f/volumes/kubernetes.io~projected/kube-api-access-89c5l major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/10c86ffc-dbfb-48d8-a225-edc41c719c6f/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/10c86ffc-dbfb-48d8-a225-edc41c719c6f/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/211b8165-d716-4097-b371-28ac08591b51/volumes/kubernetes.io~projected/kube-api-access-qhvbt:{mountpoint:/var/lib/kubelet/pods/211b8165-d716-4097-b371-28ac08591b51/volumes/kubernetes.io~projected/kube-api-access-qhvbt major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/211b8165-d716-4097-b371-28ac08591b51/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/211b8165-d716-4097-b371-28ac08591b51/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~projected/kube-api-access-gxsxd:{mountpoint:/var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~projected/kube-api-access-gxsxd major:0 minor:326 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:318 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:317 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:319 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/515e0d50-7bf5-476d-a84b-79e5f21f959f/volumes/kubernetes.io~projected/kube-api-access-ljk86:{mountpoint:/var/lib/kubelet/pods/515e0d50-7bf5-476d-a84b-79e5f21f959f/volumes/kubernetes.io~projected/kube-api-access-ljk86 major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/62af32b1-6029-4da7-b299-0d3ba5746dc6/volumes/kubernetes.io~projected/kube-api-access-lgnz9:{mountpoint:/var/lib/kubelet/pods/62af32b1-6029-4da7-b299-0d3ba5746dc6/volumes/kubernetes.io~projected/kube-api-access-lgnz9 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/653cd0d6-b7a2-446a-b677-406e3e8e56ce/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/653cd0d6-b7a2-446a-b677-406e3e8e56ce/volumes/kubernetes.io~secret/agent-certs major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/66d962d8-e0a3-4c4e-b310-e19aa5360593/volumes/kubernetes.io~projected/kube-api-access-mct7c:{mountpoint:/var/lib/kubelet/pods/66d962d8-e0a3-4c4e-b310-e19aa5360593/volumes/kubernetes.io~projected/kube-api-access-mct7c major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d74e018-659b-4322-a280-b207fed7271f/volumes/kubernetes.io~projected/kube-api-access-tkhm6:{mountpoint:/var/lib/kubelet/pods/6d74e018-659b-4322-a280-b207fed7271f/volumes/kubernetes.io~projected/kube-api-access-tkhm6 major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d74e018-659b-4322-a280-b207fed7271f/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/6d74e018-659b-4322-a280-b207fed7271f/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:276 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d74e018-659b-4322-a280-b207fed7271f/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/6d74e018-659b-4322-a280-b207fed7271f/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:275 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0/volumes/kubernetes.io~projected/kube-api-access-xgltv:{mountpoint:/var/lib/kubelet/pods/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0/volumes/kubernetes.io~projected/kube-api-access-xgltv major:0 minor:278 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:274 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:273 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:369 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c315adb-397e-4418-b193-74697cbd6b80/volumes/kubernetes.io~projected/kube-api-access-6v8wz:{mountpoint:/var/lib/kubelet/pods/7c315adb-397e-4418-b193-74697cbd6b80/volumes/kubernetes.io~projected/kube-api-access-6v8wz major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c315adb-397e-4418-b193-74697cbd6b80/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/7c315adb-397e-4418-b193-74697cbd6b80/volumes/kubernetes.io~secret/metrics-certs major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7cec808e-f1bb-4d43-98da-5779799ae953/volumes/kubernetes.io~projected/kube-api-access-sf4l6:{mountpoint:/var/lib/kubelet/pods/7cec808e-f1bb-4d43-98da-5779799ae953/volumes/kubernetes.io~projected/kube-api-access-sf4l6 major:0 minor:383 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7cec808e-f1bb-4d43-98da-5779799ae953/volumes/kubernetes.io~secret/federate-client-tls:{mountpoint:/var/lib/kubelet/pods/7cec808e-f1bb-4d43-98da-5779799ae953/volumes/kubernetes.io~secret/federate-client-tls major:0 minor:381 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7cec808e-f1bb-4d43-98da-5779799ae953/volumes/kubernetes.io~secret/secret-telemeter-client:{mountpoint:/var/lib/kubelet/pods/7cec808e-f1bb-4d43-98da-5779799ae953/volumes/kubernetes.io~secret/secret-telemeter-client major:0 minor:379 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7cec808e-f1bb-4d43-98da-5779799ae953/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/7cec808e-f1bb-4d43-98da-5779799ae953/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config major:0 minor:382 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7cec808e-f1bb-4d43-98da-5779799ae953/volumes/kubernetes.io~secret/telemeter-client-tls:{mountpoint:/var/lib/kubelet/pods/7cec808e-f1bb-4d43-98da-5779799ae953/volumes/kubernetes.io~secret/telemeter-client-tls major:0 minor:380 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7eac1f43-42ac-4e65-be08-ca44155cee2d/volumes/kubernetes.io~projected/kube-api-access-xh6cl:{mountpoint:/var/lib/kubelet/pods/7eac1f43-42ac-4e65-be08-ca44155cee2d/volumes/kubernetes.io~projected/kube-api-access-xh6cl major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7eac1f43-42ac-4e65-be08-ca44155cee2d/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/7eac1f43-42ac-4e65-be08-ca44155cee2d/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7eac1f43-42ac-4e65-be08-ca44155cee2d/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/7eac1f43-42ac-4e65-be08-ca44155cee2d/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8ec5f65f-9ac2-4ad3-812f-781cfd89d855/volumes/kubernetes.io~projected/kube-api-access-qfzpz:{mountpoint:/var/lib/kubelet/pods/8ec5f65f-9ac2-4ad3-812f-781cfd89d855/volumes/kubernetes.io~projected/kube-api-access-qfzpz major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8ec5f65f-9ac2-4ad3-812f-781cfd89d855/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/8ec5f65f-9ac2-4ad3-812f-781cfd89d855/volumes/kubernetes.io~secret/metrics-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9891960e-4422-4301-a5c4-404bff9979f9/volumes/kubernetes.io~projected/kube-api-access-5gv9f:{mountpoint:/var/lib/kubelet/pods/9891960e-4422-4301-a5c4-404bff9979f9/volumes/kubernetes.io~projected/kube-api-access-5gv9f major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9891960e-4422-4301-a5c4-404bff9979f9/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/9891960e-4422-4301-a5c4-404bff9979f9/volumes/kubernetes.io~secret/cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a414991b-4e9e-42e7-be00-a2b460ea3b2b/volumes/kubernetes.io~projected/kube-api-access-b7dn7:{mountpoint:/var/lib/kubelet/pods/a414991b-4e9e-42e7-be00-a2b460ea3b2b/volumes/kubernetes.io~projected/kube-api-access-b7dn7 major:0 minor:372 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:476 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~empty-dir/config-out major:0 minor:485 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~projected/kube-api-access-bh5mn:{mountpoint:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~projected/kube-api-access-bh5mn major:0 minor:490 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~projected/tls-assets major:0 minor:481 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/config major:0 minor:484 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:488 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:483 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:486 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:487 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:482 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:480 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:489 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/web-config major:0 minor:476 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c8bebbb6-d965-4af2-91ff-09497cdbe81e/volumes/kubernetes.io~projected/kube-api-access-m6lr5:{mountpoint:/var/lib/kubelet/pods/c8bebbb6-d965-4af2-91ff-09497cdbe81e/volumes/kubernetes.io~projected/kube-api-access-m6lr5 major:0 minor:270 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c8bebbb6-d965-4af2-91ff-09497cdbe81e/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/c8bebbb6-d965-4af2-91ff-09497cdbe81e/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:269 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c8bebbb6-d965-4af2-91ff-09497cdbe81e/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/c8bebbb6-d965-4af2-91ff-09497cdbe81e/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:265 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf8fa689-675c-4153-a857-07894dba2fe6/volumes/kubernetes.io~projected/kube-api-access-2bg9c:{mountpoint:/var/lib/kubelet/pods/cf8fa689-675c-4153-a857-07894dba2fe6/volumes/kubernetes.io~projected/kube-api-access-2bg9c major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f122e9a3-7599-48d2-b5fc-1d373d7cf446/volumes/kubernetes.io~projected/kube-api-access-46nlc:{mountpoint:/var/lib/kubelet/pods/f122e9a3-7599-48d2-b5fc-1d373d7cf446/volumes/kubernetes.io~projected/kube-api-access-46nlc major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f305ea8d-018d-4c14-a102-de41495bf1a9/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/f305ea8d-018d-4c14-a102-de41495bf1a9/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f305ea8d-018d-4c14-a102-de41495bf1a9/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/f305ea8d-018d-4c14-a102-de41495bf1a9/volumes/kubernetes.io~empty-dir/tmp major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f305ea8d-018d-4c14-a102-de41495bf1a9/volumes/kubernetes.io~projected/kube-api-access-m5zh5:{mountpoint:/var/lib/kubelet/pods/f305ea8d-018d-4c14-a102-de41495bf1a9/volumes/kubernetes.io~projected/kube-api-access-m5zh5 major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec/volumes/kubernetes.io~projected/kube-api-access-z4d2c:{mountpoint:/var/lib/kubelet/pods/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec/volumes/kubernetes.io~projected/kube-api-access-z4d2c major:0 minor:357 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:356 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/6869dac065d93d894664488d365dd8cb4348a2f4213ebca83d5332840aaab601/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/2a7a2b1555c1a2bf47680a5dfe8c1a351df4d6c5a5b36d83ffb7d6b51e42c113/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/77631aad75feb9853b763a300b5711e36d13e1675e470b4f65be45012c313e46/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/b5b92f2c00ba3c858e022d971609ed967f2c76692f934e9449764bef3cbd9232/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/c12a0f7d0704c1753ea5c0188b0048439f27c6fba90bc6a1d22a6b512b09d3df/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/958d537c8beff87d80c855df25bf8d970cbaf9ba723fda0e874397bc67d860a6/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/d49a2b0e1a4c037d53ffc4fbab3f3ea2f6390d8f438a04806543d14b0e02fa38/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/3c74505f6c6393f986585efa752df771ba48f741ff3ab3949188a40c6acaacd2/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/23fe2e3f3b28440eba66aedff75a199783fa1abb32ac9e7374a3412980472b87/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/ef49219f2941e93743e6d743e08a1fcc09a8b6e2be9ac55666c940184ea37db2/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/c7e9ebe0ac8c18b9dc5f8e05b318e8614fcd0bf630a436f90c25dd3e2407bb0d/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/36529d93e94327ea31019af2b075418f01e38e6cd2c7a95849774e79b188c2b2/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/2f13994e50f3b77d98a81024c4d70bd221350cf032a595c895f839ab639e1ee9/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/ea957dc41fee35436ab845839872091f987263722bb7aaa3239b33acac2e7b57/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-159:{mountpoint:/var/lib/containers/storage/overlay/6ae3dad177752662f30c880524db47e706f8f36c9af0712038a8fff0af7e4e5b/merged major:0 minor:159 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/3fa3ff98dbdb3187990c57166b9646c5fb136df1210c3752896e670563e92e76/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/1740da53650fd696cbb5c75ef0c87b042f7275d2b317e155e5f331a819e721cf/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/a576d7b5ec1e36c0626594ea4acf720ce70d450de9b108a5a7eb972a87a51544/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/8dcc22ae2b46c2577e58664bb2e9467bef1d13e2a2c18e43d2539fded2bd09a5/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/4c1e4e3f407fae241de2be9f67761292fe11ddc10329174f6381c5440ab99ecc/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/f40039efb0f934cbaa609cb05b27fcdf3e44a4aa6c326d8392edb43c77acd0b6/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/eb11ad77897ffd41aa3196865967635f9bf09cfa62d567d39aa90f4efa77fabd/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/670b438aef70f14103210c2663dd5ae7a9fab513b4b4518faae3bf6f5a07bcd4/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/366ec85f7bc91dcc312e6a7d4e78eb96284f9615a1b9727c4044bc43e9ee039c/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/569c5cfe51eabdb2437908b177f514349368c0bf714c0e0fab99c8d147fe0f8d/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/766e24448cabed708a96bca53caec03dd02e6154d1f601f8f1e17fcccf935ee4/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-233:{mountpoint:/var/lib/containers/storage/overlay/8edc1239414fe8a512ee5b94e1a63656ece8f781fc6ba4c645961b12955d5f7f/merged major:0 minor:233 fsType:overlay blockSize:0} overlay_0-235:{mountpoint:/var/lib/containers/storage/overlay/594cc77d4050db29b920b82e0b5e5a00f2ed74f416f65385bc9e48c94e92a5d3/merged major:0 minor:235 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/c3e033954960244eb02c1ecada753c76f40f1d4660a877d0a696a2876cb797ac/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-243:{mountpoint:/var/lib/containers/storage/overlay/393484dafdb67da401011ad2c91fa263cda748afe8d16c82d0ad284f61f008be/merged major:0 minor:243 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/100091ed6c5e6521a876a878abb1a77e669e3a0c2485a18555ef4afcc590e55b/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/34912f9fde4e3301a9fe726306ef565485e4c155a937d06a63449f0e63475125/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/f5dfda120050a1eac03049769ba9ea7a784432b7e88a8dcfae3c490023de063c/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-281:{mountpoint:/var/lib/containers/storage/overlay/4c369087da7dea33da4cbe8bbdcc9442cb41f098941664598c3ffddaec374a15/merged major:0 minor:281 fsType:overlay blockSize:0} overlay_0-285:{mountpoint:/var/lib/containers/storage/overlay/b18d27358f894ba1aac5daebfe65864b650348187275c0848b27c9406eae9505/merged major:0 minor:285 fsType:overlay blockSize:0} overlay_0-287:{mountpoint:/var/lib/containers/storage/overlay/5a50d7a16d3293bdec8cb9bf7acbcc983e222bce00c03dcc929a2b17824727a4/merged major:0 minor:287 fsType:overlay blockSize:0} overlay_0-289:{mountpoint:/var/lib/containers/storage/overlay/bd9b157910c3106ff5c6986a212da56b86cbe56164c58ae2c46d72ddeb3f0297/merged major:0 minor:289 fsType:overlay blockSize:0} overlay_0-295:{mountpoint:/var/lib/containers/storage/overlay/b3aee4619fe508e22a13a27fcc358c661f5bb790da548426e5bf5a4fccf52f58/merged major:0 minor:295 fsType:overlay blockSize:0} overlay_0-308:{mountpoint:/var/lib/containers/storage/overlay/9b9a341035f47a8876b0286340fbfab6742fb81c22894c0fdc28f44e105f6486/merged major:0 minor:308 fsType:overlay blockSize:0} overlay_0-323:{mountpoint:/var/lib/containers/storage/overlay/6290ee6b68f17f6376c0faf49aadd6a634e46d4dde93df8c09f9851226584db9/merged major:0 minor:323 fsType:overlay blockSize:0} overlay_0-325:{mountpoint:/var/lib/containers/storage/overlay/8e3a689f66dc6f21be9120f63bc603ec9e83aaa2488edcc18026ce6c243b6a03/merged major:0 minor:325 fsType:overlay blockSize:0} overlay_0-332:{mountpoint:/var/lib/containers/storage/overlay/8d885a56f99990b4c183bbb14a2dd7cb8e68f1e7069eb9dcaac682aee87732e6/merged major:0 minor:332 fsType:overlay blockSize:0} overlay_0-334:{mountpoint:/var/lib/containers/storage/overlay/0d2312ee1693e8f1dfeab805bbb37c005d388adb5dfa577f47e37608363c831d/merged major:0 minor:334 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/99e123a1b423f0766f30ce8fc3bb0f64c6c3adf4fb9bf929199096df374a0c6a/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/62564bd4729a09d735de03350f67f4b3b6a769c6f8c0b553eaa813e411c14797/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/9e4be060bd3a5faf0c1c3fd52f351b7eab7e55a48eb1f67c469115ca03df0f35/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/f8d19e7856b396c388738ac00d160ca5167ae84715acb9fa1677ca1338eb3364/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-370:{mountpoint:/var/lib/containers/storage/overlay/502a44e89766b65db05d46f90754d2d17fc95e81d3e254e9861d259db03dd03a/merged major:0 minor:370 fsType:overlay blockSize:0} overlay_0-386:{mountpoint:/var/lib/containers/storage/overlay/61f5ca1ce60422535fbf8937d32765223f309618ef6f44eb0abc8642f9a018d1/merged major:0 minor:386 fsType:overlay blockSize:0} overlay_0-388:{mountpoint:/var/lib/containers/storage/overlay/7d3c98fee775e0734304440d672960a4c88d62a46f30709053b7b2afb1aa5b75/merged major:0 minor:388 fsType:overlay blockSize:0} overlay_0-390:{mountpoint:/var/lib/containers/storage/overlay/874323b26cd63ef7f87391a0e0bb01c2d4c7802abcd7ff3f17b3ec5521f91863/merged major:0 minor:390 fsType:overlay blockSize:0} overlay_0-392:{mountpoint:/var/lib/containers/storage/overlay/7e8e6ccc8d28040e593c45b6bd9cbf4515d40a3f50fdda0ddc94e757b2c4349e/merged major:0 minor:392 fsType:overlay blockSize:0} overlay_0-394:{mountpoint:/var/lib/containers/storage/overlay/3be52d8c1a9c37fe1602955bb9261b6bb6fe1003903d3198a8adbfd42432c059/merged major:0 minor:394 fsType:overlay blockSize:0} overlay_0-396:{mountpoint:/var/lib/containers/storage/overlay/31e195dcafc6d98837bd0f3e2b742140b5cd47ad149a35efa051d716768bf491/merged major:0 minor:396 fsType:overlay blockSize:0} overlay_0-402:{mountpoint:/var/lib/containers/storage/overlay/2c90fabfaec5f6e348349377a413556c08548447e3c2d5473b480ae5cd22da81/merged major:0 minor:402 fsType:overlay blockSize:0} overlay_0-407:{mountpoint:/var/lib/containers/storage/overlay/0b1dbbfb34d8f92f66ddb634d972809464c12da23873c2f86520c557d8394830/merged major:0 minor:407 fsType:overlay blockSize:0} overlay_0-412:{mountpoint:/var/lib/containers/storage/overlay/fddcb0740da24508aee46c183c89ae23eac545cdd93d490bad54111738a8de31/merged major:0 minor:412 fsType:overlay blockSize:0} overlay_0-418:{mountpoint:/var/lib/containers/storage/overlay/ed0f43258912091753d2c2048ec61b98d554474f33bd6c371a04ce8649221be0/merged major:0 minor:418 fsType:overlay blockSize:0} overlay_0-423:{mountpoint:/var/lib/containers/storage/overlay/41a58d670dc7d994605aa54081b80f38dbddad4f259faebdeef9635dad3fa2ff/merged major:0 minor:423 fsType:overlay blockSize:0} overlay_0-426:{mountpoint:/var/lib/containers/storage/overlay/43cb02231dfde4377f01985c83a564fe45f5eef369df6d797cffd758383288ff/merged major:0 minor:426 fsType:overlay blockSize:0} overlay_0-430:{mountpoint:/var/lib/containers/storage/overlay/e501f3422106f1b61fd2d7aae1372d39d6696844c4f28d84d5b3151b3691855f/merged major:0 minor:430 fsType:overlay blockSize:0} overlay_0-436:{mountpoint:/var/lib/containers/storage/overlay/b7762b9219a05ee69e4b0f7d4b39548304db5ae8c325c7759346042dde78cd0b/merged major:0 minor:436 fsType:overlay blockSize:0} overlay_0-441:{mountpoint:/var/lib/containers/storage/overlay/15e3559b4d8b0133241d27da32032e95f95235f79bf329d2e269fce0e05a6a0d/merged major:0 minor:441 fsType:overlay blockSize:0} overlay_0-442:{mountpoint:/var/lib/containers/storage/overlay/4e113b9541989b395919439cc6c7dc436e8f602283e9eb1c5f981a957d28dedd/merged major:0 minor:442 fsType:overlay blockSize:0} overlay_0-450:{mountpoint:/var/lib/containers/storage/overlay/f10230ae12a2f320811c7197f2078ce7c324fd700db4d33295a3973894f747eb/merged major:0 minor:450 fsType:overlay blockSize:0} overlay_0-454:{mountpoint:/var/lib/containers/storage/overlay/48dd0d109ae10999e22a487ffb77d0b60e8901bf970abbc7283bf9f1e6f276a3/merged major:0 minor:454 fsType:overlay blockSize:0} overlay_0-463:{mountpoint:/var/lib/containers/storage/overlay/a748a44dd6407b29a3b7c4feaed10c8d3061d3d42e2aa45eb41b5cfa915ac5ac/merged major:0 minor:463 fsType:overlay blockSize:0} overlay_0-471:{mountpoint:/var/lib/containers/storage/overlay/8af63a23933e7fb8e556dfb37068e06fb8df24eacee5333f94379f5f5e805b1c/merged major:0 minor:471 fsType:overlay blockSize:0} overlay_0-493:{mountpoint:/var/lib/containers/storage/overlay/5a01c726ffe1858d01f5945844ec5c7c7378c53aa545844a068a892ff952d8cc/merged major:0 minor:493 fsType:overlay blockSize:0} overlay_0-495:{mountpoint:/var/lib/containers/storage/overlay/4eaf82ae968e51098e95940869006324d11388e01a6cadbe33998f4a920ceba4/merged major:0 minor:495 fsType:overlay blockSize:0} overlay_0-501:{mountpoint:/var/lib/containers/storage/overlay/27d4b632b1557c948d7a377fb894de40e70099305b1a4a7001af6949fcaac4ff/merged major:0 minor:501 fsType:overlay blockSize:0} overlay_0-506:{mountpoint:/var/lib/containers/storage/overlay/9601a103c91a85607f46814c36b768c45b99605753e43d120397150edc85fb40/merged major:0 minor:506 fsType:overlay blockSize:0} overlay_0-510:{mountpoint:/var/lib/containers/storage/overlay/1eed0a80108fd67ad99dfa87d6e2edda977a67bc9c5aa4d5d7b8f053b939782d/merged major:0 minor:510 fsType:overlay blockSize:0} overlay_0-513:{mountpoint:/var/lib/containers/storage/overlay/44ca8a25b3ddc965dad124a9d7426b3cf5431b111e1d46121470493ae1505d4a/merged major:0 minor:513 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/fb48be106c7d1132c332e0012f393509de121af9c66baf18ad6f5a69702e2902/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/c93bfe90a48cb0ec8c1a740e3150b063a20f227477cbb087283486bfb62d5113/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/8c152c21b687f575ab4a8cc9581db166e32adeb0474c6e54ec9e0082a6a9ec8d/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/bc6e4d5eefdeb7e5d9adefe51909631e03abe4f89c91d9443c612a6c5133e477/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-79:{mountpoint:/var/lib/containers/storage/overlay/15d99c415694f98f959ca23decdb172dde76618b68a88fadba27c1c89ef35cfb/merged major:0 minor:79 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/26dd5506ee0558ba46d0baca89ac393e8a9cdd392217eba24ae245b86c58bc83/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/705296dc46973c5ce627ea1960ba85c61547eb356a0b912cac1ed96a9bc0c901/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/476ff343320b1442ef4de10256becad0b8a0daed4f6a62e694675ba4a05eb3db/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/243ffe4e375a885859b529df5703b37feab2a3375134c1ce42565f5d5658d5ac/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/772257c5bd58e501835a9a124e0d1d9c83589aebb09b43bec5fec069d985e266/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.662682    5896 manager.go:217] Machine: {Timestamp:2026-10-30 22:09:07.662137939 +0000 UTC m=+0.073603896 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099929 MemoryCapacity:32812023808 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26d9563c835fd26b953ed410f3e2ea SystemUUID:ec26d956-3c83-5fd2-6b95-3ed410f3e2ea BootID:b44acf37-14d0-44a4-8170-5f40994a9b6d Filesystems:[{Device:/var/lib/kubelet/pods/f305ea8d-018d-4c14-a102-de41495bf1a9/volumes/kubernetes.io~projected/kube-api-access-m5zh5 DeviceMajor:0 DeviceMinor:57 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fc46b1d3dc62cfa9bde50dc66e27f1d33f92338594cd4d548c722369ac5937b2/userdata/shm DeviceMajor:0 DeviceMinor:62 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d9455e225f46284cda0e4a57b908471ea9595613fcab44132976c1b13e833fa6/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:317 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:307 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/62af32b1-6029-4da7-b299-0d3ba5746dc6/volumes/kubernetes.io~projected/kube-api-access-lgnz9 DeviceMajor:0 DeviceMinor:59 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-233 DeviceMajor:0 DeviceMinor:233 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8dc5263230917a41c46ec89a61dfb913c0342c027003cabaa30e27ac8c75f607/userdata/shm DeviceMajor:0 DeviceMinor:375 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/7cec808e-f1bb-4d43-98da-5779799ae953/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:382 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7eac1f43-42ac-4e65-be08-ca44155cee2d/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:229 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0/volumes/kubernetes.io~projected/kube-api-access-xgltv DeviceMajor:0 DeviceMinor:278 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-370 DeviceMajor:0 DeviceMinor:370 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-426 DeviceMajor:0 DeviceMinor:426 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8ec5f65f-9ac2-4ad3-812f-781cfd89d855/volumes/kubernetes.io~projected/kube-api-access-qfzpz DeviceMajor:0 DeviceMinor:202 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/09cc5ce9-e655-4872-a076-43e6686040e8/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:362 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/7c315adb-397e-4418-b193-74697cbd6b80/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:368 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/a414991b-4e9e-42e7-be00-a2b460ea3b2b/volumes/kubernetes.io~projected/kube-api-access-b7dn7 DeviceMajor:0 DeviceMinor:372 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-423 DeviceMajor:0 DeviceMinor:423 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/515e0d50-7bf5-476d-a84b-79e5f21f959f/volumes/kubernetes.io~projected/kube-api-access-ljk86 DeviceMajor:0 DeviceMinor:58 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b689a7577f03db498e71f937ce7caa93f0287304385a56c326c794b784410754/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-295 DeviceMajor:0 DeviceMinor:295 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:307 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8a3f580aa4c1472e7300336eb06559af45470b5b94940393259b0ffaa60da806/userdata/shm DeviceMajor:0 DeviceMinor:377 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-436 DeviceMajor:0 DeviceMinor:436 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/653cd0d6-b7a2-446a-b677-406e3e8e56ce/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:48 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8ec5f65f-9ac2-4ad3-812f-781cfd89d855/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:45 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/c8bebbb6-d965-4af2-91ff-09497cdbe81e/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:269 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec/volumes/kubernetes.io~projected/kube-api-access-z4d2c DeviceMajor:0 DeviceMinor:357 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/7cec808e-f1bb-4d43-98da-5779799ae953/volumes/kubernetes.io~secret/telemeter-client-tls DeviceMajor:0 DeviceMinor:380 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-463 DeviceMajor:0 DeviceMinor:463 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/43a9d1f849e1d1cac670ea77815734593a4bc933243c71535cb9504330a0ebc8/userdata/shm DeviceMajor:0 DeviceMinor:77 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:321 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d88aa6de1c3d54915397822d15f9302e62eff73ddfc59a5f5f25350602d75e9c/userdata/shm DeviceMajor:0 DeviceMinor:354 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-388 DeviceMajor:0 DeviceMinor:388 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-396 DeviceMajor:0 DeviceMinor:396 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-430 DeviceMajor:0 DeviceMinor:430 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2240a51eada2ade5538ed38ac3c92db4477f9c12b77115c515b52657a5fb91c3/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:307 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-412 DeviceMajor:0 DeviceMinor:412 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/10c86ffc-dbfb-48d8-a225-edc41c719c6f/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:50 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-159 DeviceMajor:0 DeviceMinor:159 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:301 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:319 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-334 DeviceMajor:0 DeviceMinor:334 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fc69fe88a9799c1453ea823b86041b89808fb2d7690c24d6575b31ce67652821/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3c8636df0f9c440a3e0a695864d0ae42f268c4f81bbec6c28f2d73a30e375eb5/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-308 DeviceMajor:0 DeviceMinor:308 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c95e58300838a2e5c7f3ce23e518cfb1bd869dac276d4c1b23691a4b47e74d6b/userdata/shm DeviceMajor:0 DeviceMinor:384 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-442 DeviceMajor:0 DeviceMinor:442 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-441 DeviceMajor:0 DeviceMinor:441 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:486 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-235 DeviceMajor:0 DeviceMinor:235 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6d74e018-659b-4322-a280-b207fed7271f/volumes/kubernetes.io~projected/kube-api-access-tkhm6 DeviceMajor:0 DeviceMinor:277 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~projected/kube-api-access-l7kdn DeviceMajor:0 DeviceMinor:310 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/09cc5ce9-e655-4872-a076-43e6686040e8/volumes/kubernetes.io~projected/kube-api-access-4zlf7 DeviceMajor:0 DeviceMinor:363 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-493 DeviceMajor:0 DeviceMinor:493 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f305ea8d-018d-4c14-a102-de41495bf1a9/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:49 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7eac1f43-42ac-4e65-be08-ca44155cee2d/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:225 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/c8bebbb6-d965-4af2-91ff-09497cdbe81e/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:265 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-394 DeviceMajor:0 DeviceMinor:394 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-418 DeviceMajor:0 DeviceMinor:418 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~projected/kube-api-access-bh5mn DeviceMajor:0 DeviceMinor:490 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406011904 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/7c315adb-397e-4418-b193-74697cbd6b80/volumes/kubernetes.io~projected/kube-api-access-6v8wz DeviceMajor:0 DeviceMinor:54 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/7eac1f43-42ac-4e65-be08-ca44155cee2d/volumes/kubernetes.io~projected/kube-api-access-xh6cl DeviceMajor:0 DeviceMinor:230 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-243 DeviceMajor:0 DeviceMinor:243 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f305ea8d-018d-4c14-a102-de41495bf1a9/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:47 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/cf8fa689-675c-4153-a857-07894dba2fe6/volumes/kubernetes.io~projected/kube-api-access-2bg9c DeviceMajor:0 DeviceMinor:52 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/0f6f4765-3e5c-441e-a388-8f56a6c180a8/volumes/kubernetes.io~projected/kube-api-access-xxwgg DeviceMajor:0 DeviceMinor:55 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c63c3353e20169010c60f5b5a591841e10c90a80d30f9c47821c7d32c763fed9/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/09cc5ce9-e655-4872-a076-43e6686040e8/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:360 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-285 DeviceMajor:0 DeviceMinor:285 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:314 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ada0931703a6b3795b0a74e04e016b530dc7a709a9571ab102f46c4626c9d8ac/userdata/shm DeviceMajor:0 DeviceMinor:373 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:489 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/211b8165-d716-4097-b371-28ac08591b51/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:197 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/c8bebbb6-d965-4af2-91ff-09497cdbe81e/volumes/kubernetes.io~projected/kube-api-access-m6lr5 DeviceMajor:0 DeviceMinor:270 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-390 DeviceMajor:0 DeviceMinor:390 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-510 DeviceMajor:0 DeviceMinor:510 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f122e9a3-7599-48d2-b5fc-1d373d7cf446/volumes/kubernetes.io~projected/kube-api-access-46nlc DeviceMajor:0 DeviceMinor:56 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:304 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:476 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:480 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:485 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:487 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-501 DeviceMajor:0 DeviceMinor:501 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/10c86ffc-dbfb-48d8-a225-edc41c719c6f/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:305 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-392 DeviceMajor:0 DeviceMinor:392 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-471 DeviceMajor:0 DeviceMinor:471 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:274 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-325 DeviceMajor:0 DeviceMinor:325 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7cec808e-f1bb-4d43-98da-5779799ae953/volumes/kubernetes.io~projected/kube-api-access-sf4l6 DeviceMajor:0 DeviceMinor:383 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:484 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:476 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eb7a959b006264436cfb665fe0dc8c1d95f5708fe651d147affac90525cdb86c/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/9891960e-4422-4301-a5c4-404bff9979f9/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:201 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7c1405656695dd347a4c305286ea14ad41d060a3efd2a1f7ce26df51bb4f26d1/userdata/shm DeviceMajor:0 DeviceMinor:491 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-79 DeviceMajor:0 DeviceMinor:79 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6d74e018-659b-4322-a280-b207fed7271f/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:275 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-281 DeviceMajor:0 DeviceMinor:281 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-332 DeviceMajor:0 DeviceMinor:332 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:369 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fcadf27badaf98b1118d9dfdc884d052646e6f29b678f6a1a7c14fb9a0ea768f/userdata/shm DeviceMajor:0 DeviceMinor:271 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/6d74e018-659b-4322-a280-b207fed7271f/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:276 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-287 DeviceMajor:0 DeviceMinor:287 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-289 DeviceMajor:0 DeviceMinor:289 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:320 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:322 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/805374b844f5cde99fd270dd7a7fc0fe4ad77396cdc9175d9b6bb73195eeffb3/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/245de05057f59faa3156b1eef737ac16096a75da62f5ec56f269a2dadb32d14e/userdata/shm DeviceMajor:0 DeviceMinor:279 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/52876bb9d9c8bfdd32770534e2826f94a7495a77322575d113bd25cafd7088b9/userdata/shm DeviceMajor:0 DeviceMinor:315 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~projected/kube-api-access-gxsxd DeviceMajor:0 DeviceMinor:326 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:356 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/48ff35a9d70b41cb87e6635608260b6f2b6d3ed3f4e72ce008240589df6e23de/userdata/shm DeviceMajor:0 DeviceMinor:358 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-454 DeviceMajor:0 DeviceMinor:454 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:481 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:483 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-506 DeviceMajor:0 DeviceMinor:506 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/09cc5ce9-e655-4872-a076-43e6686040e8/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:361 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/7cec808e-f1bb-4d43-98da-5779799ae953/volumes/kubernetes.io~secret/secret-telemeter-client DeviceMajor:0 DeviceMinor:379 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-407 DeviceMajor:0 DeviceMinor:407 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/10c86ffc-dbfb-48d8-a225-edc41c719c6f/volumes/kubernetes.io~projected/kube-api-access-89c5l DeviceMajor:0 DeviceMinor:51 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/66d962d8-e0a3-4c4e-b310-e19aa5360593/volumes/kubernetes.io~projected/kube-api-access-mct7c DeviceMajor:0 DeviceMinor:53 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4a1d5c898a7bc9ead0f87d7e30de621dba5ea85c36aa89a107ba490375462c36/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8d100abf55cb3fe20a2a2b5801b327a2a24eed9f97058ccb937ec20759f0c711/userdata/shm DeviceMajor:0 DeviceMinor:283 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9891960e-4422-4301-a5c4-404bff9979f9/volumes/kubernetes.io~projected/kube-api-access-5gv9f DeviceMajor:0 DeviceMinor:203 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1906a2e4ecb2c11646ebb3d479ba3fa31f8770eb89b919cc6a2f75583f580a1f/userdata/shm DeviceMajor:0 DeviceMinor:231 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:302 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:306 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/770d47fbb4becb4f94c63ed95394ad07136db453fae97b37023d357408eac74d/userdata/shm DeviceMajor:0 DeviceMinor:364 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:307 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-495 DeviceMajor:0 DeviceMinor:495 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/960b2d65f6ce6ab539de3b3e43c196062928b3c35558d96550ae8ba45854fc6a/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:273 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:300 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:303 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-450 DeviceMajor:0 DeviceMinor:450 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-513 DeviceMajor:0 DeviceMinor:513 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/414ea994-62c2-4250-9c5a-787cddd4b61e/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:318 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/7cec808e-f1bb-4d43-98da-5779799ae953/volumes/kubernetes.io~secret/federate-client-tls DeviceMajor:0 DeviceMinor:381 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/88936199bf332ffa78567925b302310cff7b32634bfa61ef0ce08d322819056b/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:304 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-323 DeviceMajor:0 DeviceMinor:323 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-386 DeviceMajor:0 DeviceMinor:386 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:482 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:488 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/211b8165-d716-4097-b371-28ac08591b51/volumes/kubernetes.io~projected/kube-api-access-qhvbt DeviceMajor:0 DeviceMinor:204 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-402 DeviceMajor:0 DeviceMinor:402 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:1906a2e4ecb2c11 MacAddress:32:d3:6b:4a:c5:d7 Speed:10000 Mtu:8901} {Name:2240a51eada2ade MacAddress:da:36:68:25:f7:c7 Speed:10000 Mtu:8901} {Name:48ff35a9d70b41c MacAddress:02:91:b4:3f:a5:07 Speed:10000 Mtu:8901} {Name:52876bb9d9c8bfd MacAddress:b6:61:f3:84:51:b4 Speed:10000 Mtu:8901} {Name:770d47fbb4becb4 MacAddress:f2:0c:87:f4:f5:03 Speed:10000 Mtu:8901} {Name:7c1405656695dd3 MacAddress:ea:ca:b8:41:2a:d5 Speed:10000 Mtu:8901} {Name:8a3f580aa4c1472 MacAddress:52:a9:e6:9f:fa:0d Speed:10000 Mtu:8901} {Name:8d100abf55cb3fe MacAddress:da:a4:c2:50:ef:38 Speed:10000 Mtu:8901} {Name:8dc5263230917a4 MacAddress:62:38:63:07:8b:5e Speed:10000 Mtu:8901} {Name:ada0931703a6b37 MacAddress:7e:ae:e0:21:34:4a Speed:10000 Mtu:8901} {Name:b689a7577f03db4 MacAddress:6a:15:54:93:41:e0 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:0a:44:f2:1c:59 Speed:0 Mtu:9001} {Name:br-int MacAddress:c2:d7:c9:09:3b:65 Speed:0 Mtu:8901} {Name:c95e58300838a2e MacAddress:8e:35:77:d8:24:e1 Speed:10000 Mtu:8901} {Name:d88aa6de1c3d549 MacAddress:a2:4c:2e:25:86:3c Speed:10000 Mtu:8901} {Name:d9455e225f46284 MacAddress:de:4f:f7:2d:e0:75 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:0a:44:f2:1c:59 Speed:0 Mtu:9001} {Name:fcadf27badaf98b MacAddress:6e:5d:45:5c:46:95 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:42:5a:af:06:b0:2f Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:76:f8:ee:60:a4:75 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812023808 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.663227    5896 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.663284    5896 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.663598    5896 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.663618    5896 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-202.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.663806    5896 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.663815    5896 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.663825    5896 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.663843    5896 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.663956    5896 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.664136    5896 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.664263    5896 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.664311    5896 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.664329    5896 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.664341    5896 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.664354    5896 kubelet.go:405] "Adding apiserver pod source"
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.664364    5896 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.665238    5896 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 22:09:07.665308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.665249    5896 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 22:09:07.666420 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.665710    5896 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 22:09:07.685259 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685238    5896 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 22:09:07.685259 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685257    5896 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 22:09:07.685412 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685375    5896 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 22:09:07.685601 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685582    5896 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 22:09:07.685739 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685625    5896 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 22:09:07.685807 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685759    5896 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 22:09:07.685807 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685778    5896 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 22:09:07.685807 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685789    5896 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 22:09:07.685807 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685798    5896 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 22:09:07.685807 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685807    5896 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 22:09:07.686031 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685816    5896 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 22:09:07.686031 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685826    5896 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 22:09:07.686031 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685835    5896 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 22:09:07.686031 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685845    5896 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 22:09:07.686031 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685855    5896 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 22:09:07.686031 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685869    5896 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 22:09:07.686031 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685884    5896 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 22:09:07.686031 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685918    5896 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 22:09:07.686031 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.685929    5896 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 22:09:07.689869 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.689851    5896 server.go:1263] "Started kubelet"
Oct 30 22:09:07.690066 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.690018    5896 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 22:09:07.690396 ip-10-0-138-202 systemd[1]: Started Kubernetes Kubelet.
Oct 30 22:09:07.691159 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.690598    5896 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 22:09:07.691282 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.691184    5896 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 22:09:07.691282 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.691216    5896 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 22:09:07.692396 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.692377    5896 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 22:09:07.696478 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.696313    5896 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.696872    5896 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:09:07.696915    5896 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-202.us-west-1.compute.internal\" not found"
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.697037    5896 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 22:02:55 +0000 UTC" deadline="2028-07-06 00:56:08.954606988 +0000 UTC"
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.697074    5896 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14738h47m1.25753624s"
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.696817    5896 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.697150    5896 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.697219    5896 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.699157    5896 factory.go:55] Registering systemd factory
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.699169    5896 factory.go:223] Registration of the systemd container factory successfully
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.699347    5896 factory.go:153] Registering CRI-O factory
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.699358    5896 factory.go:223] Registration of the crio container factory successfully
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.699402    5896 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.699415    5896 factory.go:103] Registering Raw factory
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.699424    5896 manager.go:1199] Started watching for new ooms in manager
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.699707    5896 manager.go:319] Starting recovery of all containers
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701210    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" volumeName="kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-cluster-tls-config" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701255    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10c86ffc-dbfb-48d8-a225-edc41c719c6f" volumeName="kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-env-overrides" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701272    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d74e018-659b-4322-a280-b207fed7271f" volumeName="kubernetes.io/secret/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701281    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ec5f65f-9ac2-4ad3-812f-781cfd89d855" volumeName="kubernetes.io/configmap/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-config-volume" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701293    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="515e0d50-7bf5-476d-a84b-79e5f21f959f" volumeName="kubernetes.io/projected/515e0d50-7bf5-476d-a84b-79e5f21f959f-kube-api-access-ljk86" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701300    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7759bd31-01d0-41bf-a1ee-4a9ddb8221d0" volumeName="kubernetes.io/empty-dir/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-textfile" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701309    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a414991b-4e9e-42e7-be00-a2b460ea3b2b" volumeName="kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701320    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701334    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f305ea8d-018d-4c14-a102-de41495bf1a9" volumeName="kubernetes.io/empty-dir/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-tuned" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701346    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="09cc5ce9-e655-4872-a076-43e6686040e8" volumeName="kubernetes.io/configmap/09cc5ce9-e655-4872-a076-43e6686040e8-metrics-server-audit-profiles" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701357    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" volumeName="kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-web-config" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701368    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cec808e-f1bb-4d43-98da-5779799ae953" volumeName="kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701375    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-web-config" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701382    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" volumeName="kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701391    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" volumeName="kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701400    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66d962d8-e0a3-4c4e-b310-e19aa5360593" volumeName="kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-cni-binary-copy" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701413    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d74e018-659b-4322-a280-b207fed7271f" volumeName="kubernetes.io/empty-dir/6d74e018-659b-4322-a280-b207fed7271f-volume-directive-shadow" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701428    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d74e018-659b-4322-a280-b207fed7271f" volumeName="kubernetes.io/projected/6d74e018-659b-4322-a280-b207fed7271f-kube-api-access-tkhm6" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701435    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7eac1f43-42ac-4e65-be08-ca44155cee2d" volumeName="kubernetes.io/projected/7eac1f43-42ac-4e65-be08-ca44155cee2d-kube-api-access-xh6cl" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701443    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="09cc5ce9-e655-4872-a076-43e6686040e8" volumeName="kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-secret-metrics-server-tls" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701449    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10c86ffc-dbfb-48d8-a225-edc41c719c6f" volumeName="kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovnkube-config" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701456    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="414ea994-62c2-4250-9c5a-787cddd4b61e" volumeName="kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-grpc-tls" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701465    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-metrics-client-certs" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701473    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="414ea994-62c2-4250-9c5a-787cddd4b61e" volumeName="kubernetes.io/configmap/414ea994-62c2-4250-9c5a-787cddd4b61e-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701485    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="414ea994-62c2-4250-9c5a-787cddd4b61e" volumeName="kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-tls" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701497    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="515e0d50-7bf5-476d-a84b-79e5f21f959f" volumeName="kubernetes.io/configmap/515e0d50-7bf5-476d-a84b-79e5f21f959f-serviceca" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701509    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c315adb-397e-4418-b193-74697cbd6b80" volumeName="kubernetes.io/projected/7c315adb-397e-4418-b193-74697cbd6b80-kube-api-access-6v8wz" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701520    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-1" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701546    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc8d2282-fc99-44d1-9d74-00eb93c5a7ec" volumeName="kubernetes.io/secret/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec-monitoring-plugin-cert" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701557    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0f6f4765-3e5c-441e-a388-8f56a6c180a8" volumeName="kubernetes.io/projected/0f6f4765-3e5c-441e-a388-8f56a6c180a8-kube-api-access-xxwgg" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701569    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="211b8165-d716-4097-b371-28ac08591b51" volumeName="kubernetes.io/secret/211b8165-d716-4097-b371-28ac08591b51-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701587    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="414ea994-62c2-4250-9c5a-787cddd4b61e" volumeName="kubernetes.io/projected/414ea994-62c2-4250-9c5a-787cddd4b61e-kube-api-access-gxsxd" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701599    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="653cd0d6-b7a2-446a-b677-406e3e8e56ce" volumeName="kubernetes.io/configmap/653cd0d6-b7a2-446a-b677-406e3e8e56ce-konnectivity-ca" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701610    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66d962d8-e0a3-4c4e-b310-e19aa5360593" volumeName="kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701622    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d74e018-659b-4322-a280-b207fed7271f" volumeName="kubernetes.io/configmap/6d74e018-659b-4322-a280-b207fed7271f-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701633    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f305ea8d-018d-4c14-a102-de41495bf1a9" volumeName="kubernetes.io/empty-dir/f305ea8d-018d-4c14-a102-de41495bf1a9-tmp" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701660    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="211b8165-d716-4097-b371-28ac08591b51" volumeName="kubernetes.io/configmap/211b8165-d716-4097-b371-28ac08591b51-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701671    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" volumeName="kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701682    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" volumeName="kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-main-tls" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701694    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="414ea994-62c2-4250-9c5a-787cddd4b61e" volumeName="kubernetes.io/empty-dir/414ea994-62c2-4250-9c5a-787cddd4b61e-tmp" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701706    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="414ea994-62c2-4250-9c5a-787cddd4b61e" volumeName="kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701717    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ec5f65f-9ac2-4ad3-812f-781cfd89d855" volumeName="kubernetes.io/projected/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-kube-api-access-qfzpz" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701729    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701740    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-grpc-tls" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701751    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" volumeName="kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-kube-api-access-l7kdn" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701762    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701774    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" volumeName="kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701785    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="414ea994-62c2-4250-9c5a-787cddd4b61e" volumeName="kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701795    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7759bd31-01d0-41bf-a1ee-4a9ddb8221d0" volumeName="kubernetes.io/configmap/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701807    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf8fa689-675c-4153-a857-07894dba2fe6" volumeName="kubernetes.io/configmap/cf8fa689-675c-4153-a857-07894dba2fe6-iptables-alerter-script" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701819    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="414ea994-62c2-4250-9c5a-787cddd4b61e" volumeName="kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701831    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7759bd31-01d0-41bf-a1ee-4a9ddb8221d0" volumeName="kubernetes.io/projected/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-kube-api-access-xgltv" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701844    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c8bebbb6-d965-4af2-91ff-09497cdbe81e" volumeName="kubernetes.io/projected/c8bebbb6-d965-4af2-91ff-09497cdbe81e-kube-api-access-m6lr5" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701856    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7eac1f43-42ac-4e65-be08-ca44155cee2d" volumeName="kubernetes.io/secret/7eac1f43-42ac-4e65-be08-ca44155cee2d-prometheus-operator-tls" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701868    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ec5f65f-9ac2-4ad3-812f-781cfd89d855" volumeName="kubernetes.io/secret/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-metrics-tls" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701882    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-tls-assets" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701903    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" volumeName="kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-out" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701915    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cec808e-f1bb-4d43-98da-5779799ae953" volumeName="kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-telemeter-client-tls" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701928    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-tls" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701942    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701953    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="09cc5ce9-e655-4872-a076-43e6686040e8" volumeName="kubernetes.io/empty-dir/09cc5ce9-e655-4872-a076-43e6686040e8-audit-log" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701966    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" volumeName="kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-volume" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701977    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="653cd0d6-b7a2-446a-b677-406e3e8e56ce" volumeName="kubernetes.io/secret/653cd0d6-b7a2-446a-b677-406e3e8e56ce-agent-certs" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.701991    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf8fa689-675c-4153-a857-07894dba2fe6" volumeName="kubernetes.io/projected/cf8fa689-675c-4153-a857-07894dba2fe6-kube-api-access-2bg9c" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702003    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" volumeName="kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-main-db" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702015    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62af32b1-6029-4da7-b299-0d3ba5746dc6" volumeName="kubernetes.io/projected/62af32b1-6029-4da7-b299-0d3ba5746dc6-kube-api-access-lgnz9" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702029    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9891960e-4422-4301-a5c4-404bff9979f9" volumeName="kubernetes.io/secret/9891960e-4422-4301-a5c4-404bff9979f9-cert" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702040    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-config-out" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702050    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc8d2282-fc99-44d1-9d74-00eb93c5a7ec" volumeName="kubernetes.io/projected/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec-kube-api-access-z4d2c" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702062    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" volumeName="kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-tls-assets" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702075    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10c86ffc-dbfb-48d8-a225-edc41c719c6f" volumeName="kubernetes.io/secret/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702086    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7759bd31-01d0-41bf-a1ee-4a9ddb8221d0" volumeName="kubernetes.io/configmap/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702097    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c8bebbb6-d965-4af2-91ff-09497cdbe81e" volumeName="kubernetes.io/secret/c8bebbb6-d965-4af2-91ff-09497cdbe81e-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702109    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-config" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702120    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="09cc5ce9-e655-4872-a076-43e6686040e8" volumeName="kubernetes.io/projected/09cc5ce9-e655-4872-a076-43e6686040e8-kube-api-access-4zlf7" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702131    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10c86ffc-dbfb-48d8-a225-edc41c719c6f" volumeName="kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702144    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10c86ffc-dbfb-48d8-a225-edc41c719c6f" volumeName="kubernetes.io/projected/10c86ffc-dbfb-48d8-a225-edc41c719c6f-kube-api-access-89c5l" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702156    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7759bd31-01d0-41bf-a1ee-4a9ddb8221d0" volumeName="kubernetes.io/secret/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702167    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="09cc5ce9-e655-4872-a076-43e6686040e8" volumeName="kubernetes.io/configmap/09cc5ce9-e655-4872-a076-43e6686040e8-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702178    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66d962d8-e0a3-4c4e-b310-e19aa5360593" volumeName="kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702190    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9891960e-4422-4301-a5c4-404bff9979f9" volumeName="kubernetes.io/projected/9891960e-4422-4301-a5c4-404bff9979f9-kube-api-access-5gv9f" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702202    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702214    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="211b8165-d716-4097-b371-28ac08591b51" volumeName="kubernetes.io/projected/211b8165-d716-4097-b371-28ac08591b51-kube-api-access-qhvbt" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702226    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="796116fb-6bb5-4c50-8b4f-9aabe94ea8ce" volumeName="kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702238    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7eac1f43-42ac-4e65-be08-ca44155cee2d" volumeName="kubernetes.io/secret/7eac1f43-42ac-4e65-be08-ca44155cee2d-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702254    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cec808e-f1bb-4d43-98da-5779799ae953" volumeName="kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702267    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702279    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cec808e-f1bb-4d43-98da-5779799ae953" volumeName="kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-secret-telemeter-client-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702291    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62af32b1-6029-4da7-b299-0d3ba5746dc6" volumeName="kubernetes.io/empty-dir/62af32b1-6029-4da7-b299-0d3ba5746dc6-tmp-dir" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702301    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="211b8165-d716-4097-b371-28ac08591b51" volumeName="kubernetes.io/empty-dir/211b8165-d716-4097-b371-28ac08591b51-data-volume" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702311    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d74e018-659b-4322-a280-b207fed7271f" volumeName="kubernetes.io/configmap/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702325    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7759bd31-01d0-41bf-a1ee-4a9ddb8221d0" volumeName="kubernetes.io/secret/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-tls" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702336    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cec808e-f1bb-4d43-98da-5779799ae953" volumeName="kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-federate-client-tls" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702347    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f305ea8d-018d-4c14-a102-de41495bf1a9" volumeName="kubernetes.io/projected/f305ea8d-018d-4c14-a102-de41495bf1a9-kube-api-access-m5zh5" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702360    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="414ea994-62c2-4250-9c5a-787cddd4b61e" volumeName="kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702371    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c315adb-397e-4418-b193-74697cbd6b80" volumeName="kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702382    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cec808e-f1bb-4d43-98da-5779799ae953" volumeName="kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-telemeter-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702402    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702417    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66d962d8-e0a3-4c4e-b310-e19aa5360593" volumeName="kubernetes.io/projected/66d962d8-e0a3-4c4e-b310-e19aa5360593-kube-api-access-mct7c" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702427    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cec808e-f1bb-4d43-98da-5779799ae953" volumeName="kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-secret-telemeter-client" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702439    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-db" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702450    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f122e9a3-7599-48d2-b5fc-1d373d7cf446" volumeName="kubernetes.io/configmap/f122e9a3-7599-48d2-b5fc-1d373d7cf446-cni-binary-copy" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702463    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ec5f65f-9ac2-4ad3-812f-781cfd89d855" volumeName="kubernetes.io/empty-dir/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-tmp-dir" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702473    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cec808e-f1bb-4d43-98da-5779799ae953" volumeName="kubernetes.io/projected/7cec808e-f1bb-4d43-98da-5779799ae953-kube-api-access-sf4l6" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702484    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7eac1f43-42ac-4e65-be08-ca44155cee2d" volumeName="kubernetes.io/configmap/7eac1f43-42ac-4e65-be08-ca44155cee2d-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702495    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c8bebbb6-d965-4af2-91ff-09497cdbe81e" volumeName="kubernetes.io/configmap/c8bebbb6-d965-4af2-91ff-09497cdbe81e-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702507    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c8bebbb6-d965-4af2-91ff-09497cdbe81e" volumeName="kubernetes.io/secret/c8bebbb6-d965-4af2-91ff-09497cdbe81e-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702518    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f122e9a3-7599-48d2-b5fc-1d373d7cf446" volumeName="kubernetes.io/projected/f122e9a3-7599-48d2-b5fc-1d373d7cf446-kube-api-access-46nlc" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702529    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="09cc5ce9-e655-4872-a076-43e6686040e8" volumeName="kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-client-ca-bundle" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702540    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f122e9a3-7599-48d2-b5fc-1d373d7cf446" volumeName="kubernetes.io/configmap/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-daemon-config" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702551    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-2" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702563    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-thanos-prometheus-http-client-file" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702593    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d74e018-659b-4322-a280-b207fed7271f" volumeName="kubernetes.io/secret/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-tls" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702605    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="09cc5ce9-e655-4872-a076-43e6686040e8" volumeName="kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-secret-metrics-server-client-certs" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702616    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" volumeName="kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702627    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-0" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702656    5896 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b127d4c4-4878-4467-b936-060ee2bcca83" volumeName="kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-kube-api-access-bh5mn" seLinuxMountContext=""
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702668    5896 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 22:09:07.703178 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.702675    5896 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 22:09:07.728600 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.728558    5896 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 22:09:07.729962 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.729937    5896 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 22:09:07.729962 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.729964    5896 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 22:09:07.730102 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.729987    5896 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 22:09:07.730102 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:09:07.730031    5896 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 22:09:07.745361 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.745331    5896 generic.go:356] "Generic (PLEG): container finished" podID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerID="8d07db46466466c90462241b0d4bee2de008cd750736078bcec8f5f26c371470" exitCode=0
Oct 30 22:09:07.749968 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.749941    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:09:07.753040 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.752973    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:09:07.767096 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.767069    5896 generic.go:356] "Generic (PLEG): container finished" podID="66d962d8-e0a3-4c4e-b310-e19aa5360593" containerID="8dcabc088a0dfcf078b8e932c6199630865300ffd381b25897b156c7cc176d1a" exitCode=0
Oct 30 22:09:07.767096 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.767094    5896 generic.go:356] "Generic (PLEG): container finished" podID="66d962d8-e0a3-4c4e-b310-e19aa5360593" containerID="8c2ff06730cfb9601e3ebe2eecf608b302ecdea64f6d44cfcb384484410cf787" exitCode=0
Oct 30 22:09:07.767231 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.767106    5896 generic.go:356] "Generic (PLEG): container finished" podID="66d962d8-e0a3-4c4e-b310-e19aa5360593" containerID="9075091f9954c443066f449dd7e7768a1d743a4d7aa43f621e8ad783be118a72" exitCode=0
Oct 30 22:09:07.767231 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.767116    5896 generic.go:356] "Generic (PLEG): container finished" podID="66d962d8-e0a3-4c4e-b310-e19aa5360593" containerID="2ddeeaa3ac8c3ffddb710ffd10f04e7f1286d4b38b43457a018b2465ccde49b5" exitCode=0
Oct 30 22:09:07.767231 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.767125    5896 generic.go:356] "Generic (PLEG): container finished" podID="66d962d8-e0a3-4c4e-b310-e19aa5360593" containerID="fb160548c91d457ead553fdcf734c3a6ecc04b63b88d48a81334fb946db268f1" exitCode=0
Oct 30 22:09:07.767231 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.767132    5896 generic.go:356] "Generic (PLEG): container finished" podID="66d962d8-e0a3-4c4e-b310-e19aa5360593" containerID="8c04266661e18a2ea0225a5660bde5c328c2ae8de13417c2d227fd24a051e77a" exitCode=0
Oct 30 22:09:07.787529 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.787497    5896 generic.go:356] "Generic (PLEG): container finished" podID="2c344a83145e337d085fbb4af4282fc3" containerID="d5365a3119f96fcaae660bc5060c9c6890d4e467c48d5fcaa4dcfbadb6727e2d" exitCode=0
Oct 30 22:09:07.789802 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.789779    5896 generic.go:356] "Generic (PLEG): container finished" podID="b127d4c4-4878-4467-b936-060ee2bcca83" containerID="595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe" exitCode=0
Oct 30 22:09:07.797198 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:09:07.796960    5896 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-202.us-west-1.compute.internal\" not found"
Oct 30 22:09:07.799046 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.799020    5896 generic.go:356] "Generic (PLEG): container finished" podID="7759bd31-01d0-41bf-a1ee-4a9ddb8221d0" containerID="347657abdffcd08e3e5bf9b38783965d7273422e7a7934841c36061bedf5b4f2" exitCode=0
Oct 30 22:09:07.827497 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.827480    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:09:07.827837 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.827813    5896 generic.go:356] "Generic (PLEG): container finished" podID="10c86ffc-dbfb-48d8-a225-edc41c719c6f" containerID="66a0846f42e52cec123734946c799d62e2caad3198483bcaacf0adb17f5f34fe" exitCode=1
Oct 30 22:09:07.830227 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:09:07.830208    5896 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 22:09:07.840281 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.840265    5896 manager.go:324] Recovery completed
Oct 30 22:09:07.867491 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.867466    5896 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:09:07.869603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.869576    5896 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:09:07.869747 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.869619    5896 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:09:07.869747 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.869634    5896 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:09:07.873289 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.873267    5896 cpu_manager.go:226] "Starting" policy="none"
Oct 30 22:09:07.873289 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.873285    5896 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 22:09:07.873289 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.873303    5896 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 22:09:07.873499 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.873286    5896 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:09:07.873499 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.873446    5896 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 22:09:07.873499 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.873458    5896 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 22:09:07.873499 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.873477    5896 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 22:09:07.873499 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.873484    5896 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 22:09:07.873499 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.873491    5896 policy_none.go:50] "Start"
Oct 30 22:09:07.873499 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.873503    5896 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 22:09:07.873845 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.873513    5896 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 22:09:07.873845 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.873659    5896 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 22:09:07.873845 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.873670    5896 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 22:09:07.873845 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.873679    5896 policy_none.go:44] "Start"
Oct 30 22:09:07.874770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.874751    5896 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:09:07.874881 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.874791    5896 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:09:07.874881 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.874806    5896 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:09:07.879559 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.879529    5896 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 22:09:07.883902 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.883873    5896 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:09:07.883902 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.883888    5896 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 22:09:07.884064 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.883922    5896 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:09:07.884064 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.883976    5896 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:09:07.884316 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.884301    5896 manager.go:341] "Starting Device Plugin manager"
Oct 30 22:09:07.885097 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:09:07.885076    5896 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 22:09:07.885097 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.885099    5896 server.go:87] "Starting device plugin registration server"
Oct 30 22:09:07.885422 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.885408    5896 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 22:09:07.885507 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.885425    5896 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 22:09:07.885590 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.885574    5896 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 22:09:07.885719 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.885688    5896 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 22:09:07.885719 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.885717    5896 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 22:09:07.885812 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.885722    5896 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 22:09:07.885812 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.885760    5896 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T22:09:07.885708675Z","UUID":"fc142fe5-3e48-47ac-9020-8b9a5e19e1a6","Handler":null,"Name":"","Endpoint":""}
Oct 30 22:09:07.888466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.887818    5896 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 22:09:07.888466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.887860    5896 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 22:09:07.892438 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:09:07.892416    5896 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 22:09:07.983666 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.983617    5896 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:09:07.985621 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.985594    5896 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:09:07.986764 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.986734    5896 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:09:07.986891 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.986782    5896 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:09:07.986891 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.986800    5896 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:09:07.986891 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:07.986867    5896 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.031263 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.031172    5896 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal"]
Oct 30 22:09:08.031958 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.031939    5896 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.032092 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.032017    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal" event={"ID":"2c344a83145e337d085fbb4af4282fc3","Type":"ContainerStarted","Data":"28f376441613fc954d0e14210d221f415eecfb3794179f032e85d72923cf200d"}
Oct 30 22:09:08.032092 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.032075    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal" event={"ID":"2c344a83145e337d085fbb4af4282fc3","Type":"ContainerDied","Data":"d5365a3119f96fcaae660bc5060c9c6890d4e467c48d5fcaa4dcfbadb6727e2d"}
Oct 30 22:09:08.032201 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.032095    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal" event={"ID":"2c344a83145e337d085fbb4af4282fc3","Type":"ContainerStarted","Data":"c63c3353e20169010c60f5b5a591841e10c90a80d30f9c47821c7d32c763fed9"}
Oct 30 22:09:08.032245 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.032205    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal" event={"ID":"dc53250de6efc08206be074330c1cea0","Type":"ContainerStarted","Data":"7840ffd6b583d319db66e45161f07717f90729ef5c06a1f4fa24c0fa7fa5f125"}
Oct 30 22:09:08.032245 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.032229    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal" event={"ID":"dc53250de6efc08206be074330c1cea0","Type":"ContainerStarted","Data":"eb7a959b006264436cfb665fe0dc8c1d95f5708fe651d147affac90525cdb86c"}
Oct 30 22:09:08.032245 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.032210    5896 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.102872 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.102844    5896 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.103013 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.102909    5896 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.109336 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.109312    5896 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:09:08.109458 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:09:08.109361    5896 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.112450 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.112431    5896 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:09:08.112568 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:09:08.112475    5896 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.185192 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.185157    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2c344a83145e337d085fbb4af4282fc3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"2c344a83145e337d085fbb4af4282fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.185336 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.185200    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c344a83145e337d085fbb4af4282fc3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"2c344a83145e337d085fbb4af4282fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.185336 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.185230    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc53250de6efc08206be074330c1cea0-config\") pod \"kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"dc53250de6efc08206be074330c1cea0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.286120 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.285974    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc53250de6efc08206be074330c1cea0-config\") pod \"kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"dc53250de6efc08206be074330c1cea0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.286268 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.286144    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2c344a83145e337d085fbb4af4282fc3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"2c344a83145e337d085fbb4af4282fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.288135 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.286022    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2c344a83145e337d085fbb4af4282fc3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"2c344a83145e337d085fbb4af4282fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.288135 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.287840    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c344a83145e337d085fbb4af4282fc3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"2c344a83145e337d085fbb4af4282fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.288135 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.286051    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc53250de6efc08206be074330c1cea0-config\") pod \"kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"dc53250de6efc08206be074330c1cea0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.288135 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.287912    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c344a83145e337d085fbb4af4282fc3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal\" (UID: \"2c344a83145e337d085fbb4af4282fc3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.665367 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.665297    5896 apiserver.go:52] "Watching apiserver"
Oct 30 22:09:08.772872 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.772842    5896 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 22:09:08.773680 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.773630    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dmwlw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal","openshift-image-registry/node-ca-krfnw","openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h","openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw","openshift-cluster-node-tuning-operator/tuned-57cq6","openshift-multus/network-metrics-daemon-fgbcv","openshift-multus/multus-additional-cni-plugins-pndld","kube-system/global-pull-secret-syncer-jxjd5","kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal","openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb","openshift-monitoring/telemeter-client-6cffccb95d-6vlcb","openshift-network-operator/iptables-alerter-nb5dx","kube-system/konnectivity-agent-f4lgs","openshift-dns/dns-default-q4cnn","openshift-multus/multus-cshfp","openshift-ovn-kubernetes/ovnkube-node-vcjjm","openshift-dns/node-resolver-lr7w8","openshift-monitoring/alertmanager-main-0","openshift-monitoring/node-exporter-hq4cs","openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j","openshift-network-diagnostics/network-check-target-s5qdj","openshift-ingress-canary/ingress-canary-lv7gq","openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk","openshift-monitoring/metrics-server-7474bd8d9b-c28kq","openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:09:08.799703 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.799669    5896 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 22:09:08.833913 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.833852    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-486qv\""
Oct 30 22:09:08.833913 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.833866    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 22:09:08.833913 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.833898    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xjxjz\""
Oct 30 22:09:08.833913 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.833897    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-mmvq7\""
Oct 30 22:09:08.834218 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.833924    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2wgjn\""
Oct 30 22:09:08.834218 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.834021    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-78xqf\""
Oct 30 22:09:08.834218 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.834042    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lmcdt\""
Oct 30 22:09:08.834218 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.834109    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 22:09:08.834218 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.834215    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tmrfp\""
Oct 30 22:09:08.834462 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.834264    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 22:09:08.834462 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.834321    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 22:09:08.834462 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.834336    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-2djd4\""
Oct 30 22:09:08.834462 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.834415    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-c8f2h\""
Oct 30 22:09:08.834681 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.834488    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-f5pfp\""
Oct 30 22:09:08.834681 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.834491    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dl5xt\""
Oct 30 22:09:08.834872 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.834857    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 22:09:08.834943 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.834932    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 22:09:08.835238 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.835221    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 22:09:08.835305 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.835221    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 22:09:08.835393 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.835376    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 22:09:08.835457 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.835408    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 22:09:08.835457 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.835426    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 22:09:08.835549 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.835479    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 22:09:08.835619 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.835600    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 22:09:08.835694 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.835604    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 22:09:08.835758 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.835727    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 22:09:08.835809 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.835763    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 22:09:08.835809 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.835774    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 22:09:08.835809 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.835716    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 22:09:08.835960 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.835816    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 22:09:08.835960 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.835855    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 22:09:08.835960 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.835911    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 22:09:08.836084 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.835992    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 22:09:08.836280 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.836263    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 22:09:08.836333 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.836290    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 22:09:08.836333 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.836295    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 22:09:08.836333 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.836327    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 22:09:08.836865 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.836849    5896 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.836915 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.836907    5896 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.891418 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891391    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-daemon-config\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.891516 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891420    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf8fa689-675c-4153-a857-07894dba2fe6-host-slash\") pod \"iptables-alerter-nb5dx\" (UID: \"cf8fa689-675c-4153-a857-07894dba2fe6\") " pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:09:08.891516 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891440    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gv9f\" (UniqueName: \"kubernetes.io/projected/9891960e-4422-4301-a5c4-404bff9979f9-kube-api-access-5gv9f\") pod \"ingress-canary-lv7gq\" (UID: \"9891960e-4422-4301-a5c4-404bff9979f9\") " pod="openshift-ingress-canary/ingress-canary-lv7gq"
Oct 30 22:09:08.891516 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891467    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.891516 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891495    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:09:08.891662 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891535    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-conf-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.891662 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891607    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46nlc\" (UniqueName: \"kubernetes.io/projected/f122e9a3-7599-48d2-b5fc-1d373d7cf446-kube-api-access-46nlc\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.891718 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891705    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfzpz\" (UniqueName: \"kubernetes.io/projected/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-kube-api-access-qfzpz\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:09:08.891718 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891713    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-daemon-config\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.891784 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891734    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-cni-multus\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.891784 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891768    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-secret-metrics-server-tls\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:08.891844 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891803    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-log-socket\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.891844 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891818    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-system-cni-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.891907 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891879    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8bebbb6-d965-4af2-91ff-09497cdbe81e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:09:08.891941 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891922    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-host\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.891980 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891955    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-secret-metrics-server-client-certs\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:08.891980 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.891975    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/09cc5ce9-e655-4872-a076-43e6686040e8-metrics-server-audit-profiles\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:08.892042 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892001    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-slash\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.892042 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892031    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.892102 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892060    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkhm6\" (UniqueName: \"kubernetes.io/projected/6d74e018-659b-4322-a280-b207fed7271f-kube-api-access-tkhm6\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:09:08.892102 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892086    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/515e0d50-7bf5-476d-a84b-79e5f21f959f-host\") pod \"node-ca-krfnw\" (UID: \"515e0d50-7bf5-476d-a84b-79e5f21f959f\") " pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:09:08.892191 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892178    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.892221 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892210    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-kubelet\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.892256 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892235    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7eac1f43-42ac-4e65-be08-ca44155cee2d-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:09:08.892331 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892318    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.892373 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892360    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:09:08.892404 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892374    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.892433 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892411    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-netns\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.892433 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892428    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-kubelet\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.892496 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892444    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-multus-certs\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.892496 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892477    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-socket-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:08.892549 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892510    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/09cc5ce9-e655-4872-a076-43e6686040e8-audit-log\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:08.892549 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892531    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.892608 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892564    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.892608 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892568    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/09cc5ce9-e655-4872-a076-43e6686040e8-audit-log\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:08.892608 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892582    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-tmp-dir\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:09:08.892705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892611    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5zh5\" (UniqueName: \"kubernetes.io/projected/f305ea8d-018d-4c14-a102-de41495bf1a9-kube-api-access-m5zh5\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.892705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892666    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-tmp-dir\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:09:08.892705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892698    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.892803 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892787    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.892847 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892830    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-run-netns\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.892886 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892870    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.892923 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892909    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mct7c\" (UniqueName: \"kubernetes.io/projected/66d962d8-e0a3-4c4e-b310-e19aa5360593-kube-api-access-mct7c\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:08.892959 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892929    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:09:08.892959 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892945    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/653cd0d6-b7a2-446a-b677-406e3e8e56ce-agent-certs\") pod \"konnectivity-agent-f4lgs\" (UID: \"653cd0d6-b7a2-446a-b677-406e3e8e56ce\") " pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:09:08.893012 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892964    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6lr5\" (UniqueName: \"kubernetes.io/projected/c8bebbb6-d965-4af2-91ff-09497cdbe81e-kube-api-access-m6lr5\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:09:08.893012 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.892988    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-kubernetes\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.893069 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893024    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysctl-conf\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.893069 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893047    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-node-log\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.893125 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893072    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.893125 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893103    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/414ea994-62c2-4250-9c5a-787cddd4b61e-tmp\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.893184 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893157    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/414ea994-62c2-4250-9c5a-787cddd4b61e-tmp\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.893184 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893170    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-k8s-cni-cncf-io\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.893245 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893193    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/653cd0d6-b7a2-446a-b677-406e3e8e56ce-konnectivity-ca\") pod \"konnectivity-agent-f4lgs\" (UID: \"653cd0d6-b7a2-446a-b677-406e3e8e56ce\") " pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:09:08.893245 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893205    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/653cd0d6-b7a2-446a-b677-406e3e8e56ce-agent-certs\") pod \"konnectivity-agent-f4lgs\" (UID: \"653cd0d6-b7a2-446a-b677-406e3e8e56ce\") " pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:09:08.893245 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893211    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljk86\" (UniqueName: \"kubernetes.io/projected/515e0d50-7bf5-476d-a84b-79e5f21f959f-kube-api-access-ljk86\") pod \"node-ca-krfnw\" (UID: \"515e0d50-7bf5-476d-a84b-79e5f21f959f\") " pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:09:08.893321 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893264    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:08.893352 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893327    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-tls\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.893413 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893399    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/653cd0d6-b7a2-446a-b677-406e3e8e56ce-konnectivity-ca\") pod \"konnectivity-agent-f4lgs\" (UID: \"653cd0d6-b7a2-446a-b677-406e3e8e56ce\") " pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:09:08.893445 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893410    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-cnibin\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:08.893445 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893435    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.893504 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893467    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-wtmp\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.893539 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893527    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-federate-client-tls\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:09:08.893569 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893563    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.893635 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893623    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v8wz\" (UniqueName: \"kubernetes.io/projected/7c315adb-397e-4418-b193-74697cbd6b80-kube-api-access-6v8wz\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:09:08.893682 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893667    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-secret-telemeter-client\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:09:08.893717 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893686    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-kubelet-config\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:09:08.893717 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893703    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:09:08.893773 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893726    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-config-volume\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:09:08.893801 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893764    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-metrics-tls\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:09:08.893861 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893846    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bg9c\" (UniqueName: \"kubernetes.io/projected/cf8fa689-675c-4153-a857-07894dba2fe6-kube-api-access-2bg9c\") pod \"iptables-alerter-nb5dx\" (UID: \"cf8fa689-675c-4153-a857-07894dba2fe6\") " pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:09:08.893938 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893880    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09cc5ce9-e655-4872-a076-43e6686040e8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:08.893938 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893932    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.894018 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893963    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:09:08.894018 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.893987    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/211b8165-d716-4097-b371-28ac08591b51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:09:08.894089 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894026    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.894089 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894050    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-run-ovn-kubernetes\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.894089 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894069    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.894089 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894084    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-out\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.894267 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894106    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62af32b1-6029-4da7-b299-0d3ba5746dc6-tmp-dir\") pod \"node-resolver-lr7w8\" (UID: \"62af32b1-6029-4da7-b299-0d3ba5746dc6\") " pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:09:08.894267 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894130    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.894267 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894128    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-original-pull-secret\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:09:08.894267 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894182    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovnkube-script-lib\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.894267 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894193    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62af32b1-6029-4da7-b299-0d3ba5746dc6-tmp-dir\") pod \"node-resolver-lr7w8\" (UID: \"62af32b1-6029-4da7-b299-0d3ba5746dc6\") " pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:09:08.894267 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894207    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-system-cni-dir\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:08.894267 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894190    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-out\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.894267 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894242    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:08.894613 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894279    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-var-lib-kubelet\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.894613 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894309    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zlf7\" (UniqueName: \"kubernetes.io/projected/09cc5ce9-e655-4872-a076-43e6686040e8-kube-api-access-4zlf7\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:08.894613 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894399    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.894613 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894440    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:08.894613 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894478    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-device-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:08.894613 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894530    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.894613 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894570    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-ovn\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.894613 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894598    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-cni-bin\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.894922 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894620    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-modprobe-d\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.894922 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894679    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.894922 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894702    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.894922 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894728    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7eac1f43-42ac-4e65-be08-ca44155cee2d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:09:08.894922 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894764    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d74e018-659b-4322-a280-b207fed7271f-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:09:08.894922 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894822    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysconfig\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.894922 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894842    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-lib-modules\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.894922 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894890    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-sys-fs\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:08.895287 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894939    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-etc-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.895287 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.894977    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:09:08.895287 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895011    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.895287 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895041    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-registration-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:08.895287 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895074    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7dn7\" (UniqueName: \"kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7\") pod \"network-check-target-s5qdj\" (UID: \"a414991b-4e9e-42e7-be00-a2b460ea3b2b\") " pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:09:08.895287 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895106    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.895287 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895138    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf4l6\" (UniqueName: \"kubernetes.io/projected/7cec808e-f1bb-4d43-98da-5779799ae953-kube-api-access-sf4l6\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:09:08.895287 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895155    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f122e9a3-7599-48d2-b5fc-1d373d7cf446-cni-binary-copy\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.895287 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895190    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-etc-kubernetes\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.895287 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895206    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8bebbb6-d965-4af2-91ff-09497cdbe81e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:09:08.895287 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895220    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/62af32b1-6029-4da7-b299-0d3ba5746dc6-hosts-file\") pod \"node-resolver-lr7w8\" (UID: \"62af32b1-6029-4da7-b299-0d3ba5746dc6\") " pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:09:08.895287 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895237    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.895287 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895268    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh5mn\" (UniqueName: \"kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-kube-api-access-bh5mn\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895338    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-tls\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895416    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-tuned\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895484    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f122e9a3-7599-48d2-b5fc-1d373d7cf446-cni-binary-copy\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895477    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f305ea8d-018d-4c14-a102-de41495bf1a9-tmp\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895487    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-tuned\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895530    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-var-lib-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895562    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovnkube-config\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895567    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f305ea8d-018d-4c14-a102-de41495bf1a9-tmp\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895579    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-sys\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895614    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-metrics-client-ca\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895680    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895706    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/515e0d50-7bf5-476d-a84b-79e5f21f959f-serviceca\") pod \"node-ca-krfnw\" (UID: \"515e0d50-7bf5-476d-a84b-79e5f21f959f\") " pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895721    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895774    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh6cl\" (UniqueName: \"kubernetes.io/projected/7eac1f43-42ac-4e65-be08-ca44155cee2d-kube-api-access-xh6cl\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895801    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-web-config\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895840    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgnz9\" (UniqueName: \"kubernetes.io/projected/62af32b1-6029-4da7-b299-0d3ba5746dc6-kube-api-access-lgnz9\") pod \"node-resolver-lr7w8\" (UID: \"62af32b1-6029-4da7-b299-0d3ba5746dc6\") " pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895867    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-sys\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895879    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovnkube-config\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895910    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895967    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9891960e-4422-4301-a5c4-404bff9979f9-cert\") pod \"ingress-canary-lv7gq\" (UID: \"9891960e-4422-4301-a5c4-404bff9979f9\") " pod="openshift-ingress-canary/ingress-canary-lv7gq"
Oct 30 22:09:08.896006 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.895996    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4d2c\" (UniqueName: \"kubernetes.io/projected/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec-kube-api-access-z4d2c\") pod \"monitoring-plugin-77f4f76594-pvxcb\" (UID: \"fc8d2282-fc99-44d1-9d74-00eb93c5a7ec\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896026    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6d74e018-659b-4322-a280-b207fed7271f-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896051    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/414ea994-62c2-4250-9c5a-787cddd4b61e-metrics-client-ca\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896077    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-grpc-tls\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896111    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-dbus\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896139    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89c5l\" (UniqueName: \"kubernetes.io/projected/10c86ffc-dbfb-48d8-a225-edc41c719c6f-kube-api-access-89c5l\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896146    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6d74e018-659b-4322-a280-b207fed7271f-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896180    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eac1f43-42ac-4e65-be08-ca44155cee2d-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896258    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/515e0d50-7bf5-476d-a84b-79e5f21f959f-serviceca\") pod \"node-ca-krfnw\" (UID: \"515e0d50-7bf5-476d-a84b-79e5f21f959f\") " pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896272    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/211b8165-d716-4097-b371-28ac08591b51-data-volume\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896296    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovn-node-metrics-cert\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896312    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-etc-selinux\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896329    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-client-ca-bundle\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896351    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-cni-bin\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896379    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896443    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhvbt\" (UniqueName: \"kubernetes.io/projected/211b8165-d716-4097-b371-28ac08591b51-kube-api-access-qhvbt\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896462    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896474    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/211b8165-d716-4097-b371-28ac08591b51-data-volume\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896532    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-env-overrides\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896601    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-pvxcb\" (UID: \"fc8d2282-fc99-44d1-9d74-00eb93c5a7ec\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896695    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-config-out\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896728    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxsxd\" (UniqueName: \"kubernetes.io/projected/414ea994-62c2-4250-9c5a-787cddd4b61e-kube-api-access-gxsxd\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.896770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896747    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-serving-certs-ca-bundle\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896832    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-config-out\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896858    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-telemeter-client-tls\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896877    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-cnibin\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896892    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-hostroot\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896906    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-cni-netd\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896956    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-root\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.896986    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8bebbb6-d965-4af2-91ff-09497cdbe81e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897013    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-config\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897077    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-systemd-units\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897122    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-systemd\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897144    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgltv\" (UniqueName: \"kubernetes.io/projected/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-kube-api-access-xgltv\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897194    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-volume\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897217    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897235    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897249    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-cni-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897266    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-run\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897284    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897328    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7kdn\" (UniqueName: \"kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-kube-api-access-l7kdn\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897353    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897382    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-cni-binary-copy\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897410    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897443    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-metrics-client-ca\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897469    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/211b8165-d716-4097-b371-28ac08591b51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897559    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c315adb-397e-4418-b193-74697cbd6b80-metrics-certs\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:09:08.897604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897586    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysctl-d\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.898466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897634    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-systemd\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.898466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897716    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxwgg\" (UniqueName: \"kubernetes.io/projected/0f6f4765-3e5c-441e-a388-8f56a6c180a8-kube-api-access-xxwgg\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:08.898466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897760    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-textfile\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.898466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897801    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.898466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897830    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/211b8165-d716-4097-b371-28ac08591b51-crio-socket\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:09:08.898466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897863    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf8fa689-675c-4153-a857-07894dba2fe6-iptables-alerter-script\") pod \"iptables-alerter-nb5dx\" (UID: \"cf8fa689-675c-4153-a857-07894dba2fe6\") " pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:09:08.898466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897894    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-textfile\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.898466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897892    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.898466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897932    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-web-config\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.898466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.897997    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66d962d8-e0a3-4c4e-b310-e19aa5360593-cni-binary-copy\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:08.898466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.898009    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.898466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.898064    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.898466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.898084    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf8fa689-675c-4153-a857-07894dba2fe6-iptables-alerter-script\") pod \"iptables-alerter-nb5dx\" (UID: \"cf8fa689-675c-4153-a857-07894dba2fe6\") " pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:09:08.898466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.898135    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-socket-dir-parent\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.898466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.898177    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.901001 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.900985    5896 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:09:08.901079 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:09:08.901024    5896 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.903122 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903105    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 22:09:08.903206 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903129    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 22:09:08.903206 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903130    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 22:09:08.903206 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903131    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 22:09:08.903488 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903437    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 22:09:08.903488 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903441    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-4ltrt\""
Oct 30 22:09:08.903620 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903595    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 22:09:08.903620 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903610    5896 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:09:08.903752 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903632    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 22:09:08.903752 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903666    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 22:09:08.903752 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903670    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 22:09:08.903752 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903701    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 22:09:08.903752 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903729    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-9wjrx\""
Oct 30 22:09:08.903752 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:09:08.903732    5896 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-202.us-west-1.compute.internal"
Oct 30 22:09:08.903752 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903747    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 22:09:08.904048 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903779    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 22:09:08.904048 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903807    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 22:09:08.904048 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903811    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 22:09:08.904048 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903842    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bq6t45e7eerjf\""
Oct 30 22:09:08.904048 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903891    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 22:09:08.904048 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.903900    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 22:09:08.904253 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.904084    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 22:09:08.904253 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.904137    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 22:09:08.904349 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.904332    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/211b8165-d716-4097-b371-28ac08591b51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:09:08.904393 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.904356    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.904550 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.904538    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 22:09:08.904624 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.904611    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-n9qnm\""
Oct 30 22:09:08.904707 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.904618    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 22:09:08.904707 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.904666    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 22:09:08.904707 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.904665    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 22:09:08.904850 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.904794    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-kpsh7\""
Oct 30 22:09:08.904949 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.904930    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 22:09:08.904949 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.904947    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sjbnz\""
Oct 30 22:09:08.905105 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905000    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qknqm\""
Oct 30 22:09:08.905105 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905027    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tc949\""
Oct 30 22:09:08.905105 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905036    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 22:09:08.905105 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.904948    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 22:09:08.905105 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905047    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 22:09:08.905105 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905089    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 22:09:08.905389 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905143    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lds9w\""
Oct 30 22:09:08.905389 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905204    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 22:09:08.905389 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905201    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7eac1f43-42ac-4e65-be08-ca44155cee2d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:09:08.905389 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905232    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.905389 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905342    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 22:09:08.905389 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905361    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nff88\""
Oct 30 22:09:08.905698 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905474    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 22:09:08.905698 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905488    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.905698 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905547    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-xsh7n\""
Oct 30 22:09:08.905698 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905563    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2fh8n\""
Oct 30 22:09:08.905698 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905577    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 22:09:08.905698 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905601    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 22:09:08.905698 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905611    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 22:09:08.905698 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905617    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 22:09:08.905698 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905611    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 22:09:08.905698 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905567    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 22:09:08.905698 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905662    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 22:09:08.905698 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905657    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 22:09:08.905698 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905693    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 22:09:08.905698 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905704    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 22:09:08.906156 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905727    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 22:09:08.906156 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.905737    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qzh2l\""
Oct 30 22:09:08.906156 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.906142    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bo7a2mdfbp4fu\""
Oct 30 22:09:08.906291 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.906265    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.906336 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.906315    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-web-config\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.906472 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.906456    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xclcc\""
Oct 30 22:09:08.906523 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.906466    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-tls\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.906523 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.906485    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-92lj7rkq13635\""
Oct 30 22:09:08.906613 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.906550    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-lsrrd\""
Oct 30 22:09:08.906682 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.906673    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9891960e-4422-4301-a5c4-404bff9979f9-cert\") pod \"ingress-canary-lv7gq\" (UID: \"9891960e-4422-4301-a5c4-404bff9979f9\") " pod="openshift-ingress-canary/ingress-canary-lv7gq"
Oct 30 22:09:08.906750 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.906731    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovn-node-metrics-cert\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.906891 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.906872    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.906948 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.906878    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.907000 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.906983    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-client-ca-bundle\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:08.907062 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.907043    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-pvxcb\" (UID: \"fc8d2282-fc99-44d1-9d74-00eb93c5a7ec\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"
Oct 30 22:09:08.907105 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.907093    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-telemeter-client-tls\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:09:08.907223 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.907206    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8bebbb6-d965-4af2-91ff-09497cdbe81e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:09:08.907555 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.907536    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-config\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.907794 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.907771    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-volume\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.907992 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.907974    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.908161 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.908143    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.908344 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.908329    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.908471 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.908456    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.908620 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.908607    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-web-config\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.911872 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.911854    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:09:08.912052 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.912030    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-secret-metrics-server-tls\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:08.912150 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.912050    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.912199 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.912170    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8bebbb6-d965-4af2-91ff-09497cdbe81e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:09:08.912245 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.912226    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/09cc5ce9-e655-4872-a076-43e6686040e8-secret-metrics-server-client-certs\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:08.913047 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.913027    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.913111 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.913050    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.913111 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.913065    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7eac1f43-42ac-4e65-be08-ca44155cee2d-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:09:08.913111 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.913053    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:09:08.913111 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.913031    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.913256 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.913183    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:09:08.913256 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.913220    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.914330 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.914313    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-secret-telemeter-client\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:09:08.914379 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.914350    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.914379 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.914357    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-metrics-tls\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:09:08.914379 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.914364    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-tls\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.914468 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.914440    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7cec808e-f1bb-4d43-98da-5779799ae953-federate-client-tls\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:09:08.914468 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.914442    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.914536 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.914485    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.915610 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.915569    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.916705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.916691    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/414ea994-62c2-4250-9c5a-787cddd4b61e-secret-grpc-tls\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.938062 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.938036    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 22:09:08.938402 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.938382    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 22:09:08.938477 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.938385    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 22:09:08.938588 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.938574    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 22:09:08.938631 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.938594    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 22:09:08.938631 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.938623    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 22:09:08.939095 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.939080    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 22:09:08.939170 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.939109    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 22:09:08.939170 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.939122    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 22:09:08.939170 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.939140    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 22:09:08.939170 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.939151    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 22:09:08.939170 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.939165    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 22:09:08.939390 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.939184    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 22:09:08.939390 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.939213    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 22:09:08.939390 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.939220    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 22:09:08.939390 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.939223    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 22:09:08.939390 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.939367    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 22:09:08.939625 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.939396    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 22:09:08.939691 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.939634    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 22:09:08.941237 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.941223    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 22:09:08.942492 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.942477    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/09cc5ce9-e655-4872-a076-43e6686040e8-metrics-server-audit-profiles\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:08.943559 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.943545    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.943602 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.943579    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.944797 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.944777    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.944878 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.944819    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09cc5ce9-e655-4872-a076-43e6686040e8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:08.944878 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.944822    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.944878 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.944839    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.944878 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.944841    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-config-volume\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:09:08.946007 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.945985    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.946120 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.946025    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6d74e018-659b-4322-a280-b207fed7271f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:09:08.946120 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.946030    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8bebbb6-d965-4af2-91ff-09497cdbe81e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:09:08.946120 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.946052    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-metrics-client-ca\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.946120 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.946061    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d74e018-659b-4322-a280-b207fed7271f-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:09:08.947174 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.947158    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-env-overrides\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.947249 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.947211    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eac1f43-42ac-4e65-be08-ca44155cee2d-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:09:08.947249 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.947214    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/414ea994-62c2-4250-9c5a-787cddd4b61e-metrics-client-ca\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.947323 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.947287    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-serving-certs-ca-bundle\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:09:08.948416 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.948401    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-metrics-client-ca\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:09:08.948461 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.948424    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/211b8165-d716-4097-b371-28ac08591b51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:09:08.948461 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.948427    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.953885 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.953856    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 22:09:08.954475 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.954457    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/10c86ffc-dbfb-48d8-a225-edc41c719c6f-ovnkube-script-lib\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.989089 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.989060    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 22:09:08.989541 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.989523    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 22:09:08.990073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.990048    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46nlc\" (UniqueName: \"kubernetes.io/projected/f122e9a3-7599-48d2-b5fc-1d373d7cf446-kube-api-access-46nlc\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.990154 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.990111    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 22:09:08.990154 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.990112    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfzpz\" (UniqueName: \"kubernetes.io/projected/8ec5f65f-9ac2-4ad3-812f-781cfd89d855-kube-api-access-qfzpz\") pod \"dns-default-q4cnn\" (UID: \"8ec5f65f-9ac2-4ad3-812f-781cfd89d855\") " pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:09:08.990270 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.990250    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 22:09:08.990327 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.990251    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljk86\" (UniqueName: \"kubernetes.io/projected/515e0d50-7bf5-476d-a84b-79e5f21f959f-kube-api-access-ljk86\") pod \"node-ca-krfnw\" (UID: \"515e0d50-7bf5-476d-a84b-79e5f21f959f\") " pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:09:08.990389 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.990366    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mct7c\" (UniqueName: \"kubernetes.io/projected/66d962d8-e0a3-4c4e-b310-e19aa5360593-kube-api-access-mct7c\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:08.990438 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.990393    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkhm6\" (UniqueName: \"kubernetes.io/projected/6d74e018-659b-4322-a280-b207fed7271f-kube-api-access-tkhm6\") pod \"kube-state-metrics-5dc8d7d89b-sz4mk\" (UID: \"6d74e018-659b-4322-a280-b207fed7271f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-sz4mk"
Oct 30 22:09:08.990860 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.990837    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zlf7\" (UniqueName: \"kubernetes.io/projected/09cc5ce9-e655-4872-a076-43e6686040e8-kube-api-access-4zlf7\") pod \"metrics-server-7474bd8d9b-c28kq\" (UID: \"09cc5ce9-e655-4872-a076-43e6686040e8\") " pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:08.990941 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.990910    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh6cl\" (UniqueName: \"kubernetes.io/projected/7eac1f43-42ac-4e65-be08-ca44155cee2d-kube-api-access-xh6cl\") pod \"prometheus-operator-9cd89b74c-4v9pw\" (UID: \"7eac1f43-42ac-4e65-be08-ca44155cee2d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4v9pw"
Oct 30 22:09:08.990941 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.990914    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5zh5\" (UniqueName: \"kubernetes.io/projected/f305ea8d-018d-4c14-a102-de41495bf1a9-kube-api-access-m5zh5\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.991091 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.990966    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxwgg\" (UniqueName: \"kubernetes.io/projected/0f6f4765-3e5c-441e-a388-8f56a6c180a8-kube-api-access-xxwgg\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:08.991283 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.991262    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6lr5\" (UniqueName: \"kubernetes.io/projected/c8bebbb6-d965-4af2-91ff-09497cdbe81e-kube-api-access-m6lr5\") pod \"openshift-state-metrics-7bd7489957-sm67h\" (UID: \"c8bebbb6-d965-4af2-91ff-09497cdbe81e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-sm67h"
Oct 30 22:09:08.991372 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.991351    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf4l6\" (UniqueName: \"kubernetes.io/projected/7cec808e-f1bb-4d43-98da-5779799ae953-kube-api-access-sf4l6\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:09:08.991427 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.991395    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7kdn\" (UniqueName: \"kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-kube-api-access-l7kdn\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.991482 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.991462    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v8wz\" (UniqueName: \"kubernetes.io/projected/7c315adb-397e-4418-b193-74697cbd6b80-kube-api-access-6v8wz\") pod \"network-metrics-daemon-fgbcv\" (UID: \"7c315adb-397e-4418-b193-74697cbd6b80\") " pod="openshift-multus/network-metrics-daemon-fgbcv"
Oct 30 22:09:08.991573 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.991552    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7dn7\" (UniqueName: \"kubernetes.io/projected/a414991b-4e9e-42e7-be00-a2b460ea3b2b-kube-api-access-b7dn7\") pod \"network-check-target-s5qdj\" (UID: \"a414991b-4e9e-42e7-be00-a2b460ea3b2b\") " pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:09:08.991632 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.991614    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gv9f\" (UniqueName: \"kubernetes.io/projected/9891960e-4422-4301-a5c4-404bff9979f9-kube-api-access-5gv9f\") pod \"ingress-canary-lv7gq\" (UID: \"9891960e-4422-4301-a5c4-404bff9979f9\") " pod="openshift-ingress-canary/ingress-canary-lv7gq"
Oct 30 22:09:08.991740 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.991720    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4d2c\" (UniqueName: \"kubernetes.io/projected/fc8d2282-fc99-44d1-9d74-00eb93c5a7ec-kube-api-access-z4d2c\") pod \"monitoring-plugin-77f4f76594-pvxcb\" (UID: \"fc8d2282-fc99-44d1-9d74-00eb93c5a7ec\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"
Oct 30 22:09:08.991820 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.991800    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgnz9\" (UniqueName: \"kubernetes.io/projected/62af32b1-6029-4da7-b299-0d3ba5746dc6-kube-api-access-lgnz9\") pod \"node-resolver-lr7w8\" (UID: \"62af32b1-6029-4da7-b299-0d3ba5746dc6\") " pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:09:08.991915 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.991893    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bg9c\" (UniqueName: \"kubernetes.io/projected/cf8fa689-675c-4153-a857-07894dba2fe6-kube-api-access-2bg9c\") pod \"iptables-alerter-nb5dx\" (UID: \"cf8fa689-675c-4153-a857-07894dba2fe6\") " pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:09:08.991990 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.991960    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhvbt\" (UniqueName: \"kubernetes.io/projected/211b8165-d716-4097-b371-28ac08591b51-kube-api-access-qhvbt\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:09:08.992044 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.992006    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgltv\" (UniqueName: \"kubernetes.io/projected/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-kube-api-access-xgltv\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.992098 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.992054    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh5mn\" (UniqueName: \"kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-kube-api-access-bh5mn\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.992151 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.992144    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxsxd\" (UniqueName: \"kubernetes.io/projected/414ea994-62c2-4250-9c5a-787cddd4b61e-kube-api-access-gxsxd\") pod \"thanos-querier-7d45fc54c5-cz6fw\" (UID: \"414ea994-62c2-4250-9c5a-787cddd4b61e\") " pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:08.992273 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.992253    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89c5l\" (UniqueName: \"kubernetes.io/projected/10c86ffc-dbfb-48d8-a225-edc41c719c6f-kube-api-access-89c5l\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.992503 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.992485    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:09:08.995236 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.995215    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.995294 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.995275    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cec808e-f1bb-4d43-98da-5779799ae953-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cffccb95d-6vlcb\" (UID: \"7cec808e-f1bb-4d43-98da-5779799ae953\") " pod="openshift-monitoring/telemeter-client-6cffccb95d-6vlcb"
Oct 30 22:09:08.996580 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.996563    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:08.998487 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998472    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-etc-kubernetes\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.998531 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998495    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/62af32b1-6029-4da7-b299-0d3ba5746dc6-hosts-file\") pod \"node-resolver-lr7w8\" (UID: \"62af32b1-6029-4da7-b299-0d3ba5746dc6\") " pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:09:08.998531 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998512    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-var-lib-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.998531 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998525    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-sys\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.998675 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998560    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-etc-kubernetes\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.998675 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998572    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-sys\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.998675 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998578    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/62af32b1-6029-4da7-b299-0d3ba5746dc6-hosts-file\") pod \"node-resolver-lr7w8\" (UID: \"62af32b1-6029-4da7-b299-0d3ba5746dc6\") " pod="openshift-dns/node-resolver-lr7w8"
Oct 30 22:09:08.998675 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998586    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-var-lib-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.998675 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998615    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-sys\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.998675 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998623    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-dbus\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:09:08.998675 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998670    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-etc-selinux\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998692    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-dbus\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998698    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-cni-bin\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998626    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-sys\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998727    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-cnibin\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998747    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-etc-selinux\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998751    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-hostroot\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998762    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-cni-bin\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998777    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-cni-netd\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998782    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-cnibin\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998799    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-root\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998814    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-cni-netd\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998801    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-hostroot\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998824    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-systemd-units\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998820    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-root\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998848    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-systemd-units\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998852    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-systemd\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998877    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-cni-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998909    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-run\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998931    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-systemd\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998933    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998946    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-cni-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998961    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-run\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998960    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998973    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.998995    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysctl-d\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999011    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:08.999008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999016    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-systemd\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999039    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-systemd\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999050    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/211b8165-d716-4097-b371-28ac08591b51-crio-socket\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999081    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-socket-dir-parent\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999106    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf8fa689-675c-4153-a857-07894dba2fe6-host-slash\") pod \"iptables-alerter-nb5dx\" (UID: \"cf8fa689-675c-4153-a857-07894dba2fe6\") " pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999133    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/211b8165-d716-4097-b371-28ac08591b51-crio-socket\") pod \"insights-runtime-extractor-dmwlw\" (UID: \"211b8165-d716-4097-b371-28ac08591b51\") " pod="openshift-insights/insights-runtime-extractor-dmwlw"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999139    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-socket-dir-parent\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999131    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysctl-d\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999132    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-conf-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999154    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf8fa689-675c-4153-a857-07894dba2fe6-host-slash\") pod \"iptables-alerter-nb5dx\" (UID: \"cf8fa689-675c-4153-a857-07894dba2fe6\") " pod="openshift-network-operator/iptables-alerter-nb5dx"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999170    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-cni-multus\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999186    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-log-socket\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999200    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-cni-multus\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999169    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-multus-conf-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999204    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-system-cni-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999227    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-system-cni-dir\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999231    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-log-socket\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999243    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-host\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999260    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-slash\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999275    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/515e0d50-7bf5-476d-a84b-79e5f21f959f-host\") pod \"node-ca-krfnw\" (UID: \"515e0d50-7bf5-476d-a84b-79e5f21f959f\") " pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999288    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-kubelet\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999312    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-slash\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999319    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-netns\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999326    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-kubelet\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999332    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-host\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999344    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-kubelet\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999349    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/515e0d50-7bf5-476d-a84b-79e5f21f959f-host\") pod \"node-ca-krfnw\" (UID: \"515e0d50-7bf5-476d-a84b-79e5f21f959f\") " pod="openshift-image-registry/node-ca-krfnw"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999363    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-multus-certs\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999385    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-multus-certs\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999390    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-socket-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999405    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-netns\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999420    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-run-netns\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999448    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999470    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-run-netns\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999475    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-kubernetes\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999422    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-socket-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999421    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-kubelet\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999501    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysctl-conf\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999511    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999515    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-kubernetes\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999523    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-node-log\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999548    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-k8s-cni-cncf-io\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999569    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-node-log\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999575    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999595    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysctl-conf\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999602    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-run-k8s-cni-cncf-io\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999602    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-cnibin\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999627    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999628    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-wtmp\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999679    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-cnibin\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999685    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-kubelet-config\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999713    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7759bd31-01d0-41bf-a1ee-4a9ddb8221d0-node-exporter-wtmp\") pod \"node-exporter-hq4cs\" (UID: \"7759bd31-01d0-41bf-a1ee-4a9ddb8221d0\") " pod="openshift-monitoring/node-exporter-hq4cs"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999716    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-run-ovn-kubernetes\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999744    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-system-cni-dir\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999749    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/796116fb-6bb5-4c50-8b4f-9aabe94ea8ce-kubelet-config\") pod \"global-pull-secret-syncer-jxjd5\" (UID: \"796116fb-6bb5-4c50-8b4f-9aabe94ea8ce\") " pod="kube-system/global-pull-secret-syncer-jxjd5"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999768    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-var-lib-kubelet\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999774    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-host-run-ovn-kubernetes\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999792    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-device-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999808    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66d962d8-e0a3-4c4e-b310-e19aa5360593-system-cni-dir\") pod \"multus-additional-cni-plugins-pndld\" (UID: \"66d962d8-e0a3-4c4e-b310-e19aa5360593\") " pod="openshift-multus/multus-additional-cni-plugins-pndld"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999814    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-ovn\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999838    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-var-lib-kubelet\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999858    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-device-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999863    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-run-ovn\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999872    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-cni-bin\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999896    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-modprobe-d\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999913    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysconfig\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999927    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-lib-modules\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999940    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-sys-fs\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999955    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-etc-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999939    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f122e9a3-7599-48d2-b5fc-1d373d7cf446-host-var-lib-cni-bin\") pod \"multus-cshfp\" (UID: \"f122e9a3-7599-48d2-b5fc-1d373d7cf446\") " pod="openshift-multus/multus-cshfp"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999978    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-registration-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999984    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-sys-fs\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:08.999985    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-sysconfig\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:09.000013    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-etc-modprobe-d\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:09.000020    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f305ea8d-018d-4c14-a102-de41495bf1a9-lib-modules\") pod \"tuned-57cq6\" (UID: \"f305ea8d-018d-4c14-a102-de41495bf1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-57cq6"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:09.000013    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10c86ffc-dbfb-48d8-a225-edc41c719c6f-etc-openvswitch\") pod \"ovnkube-node-vcjjm\" (UID: \"10c86ffc-dbfb-48d8-a225-edc41c719c6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:09.000094 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:09.000040    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f6f4765-3e5c-441e-a388-8f56a6c180a8-registration-dir\") pod \"aws-ebs-csi-driver-node-xr89j\" (UID: \"0f6f4765-3e5c-441e-a388-8f56a6c180a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xr89j"
Oct 30 22:09:09.091034 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:09.091018    5896 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:09:10.592698 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:10.592603    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:10.598396 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:10.598373    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7d45fc54c5-cz6fw"
Oct 30 22:09:10.638416 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:10.638392    5896 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:10.854919 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:10.854837    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b127d4c4-4878-4467-b936-060ee2bcca83","Type":"ContainerStarted","Data":"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525"}
Oct 30 22:09:10.855073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:10.854950    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b127d4c4-4878-4467-b936-060ee2bcca83","Type":"ContainerStarted","Data":"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d"}
Oct 30 22:09:10.855073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:10.854986    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b127d4c4-4878-4467-b936-060ee2bcca83","Type":"ContainerStarted","Data":"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916"}
Oct 30 22:09:10.855073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:10.855011    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b127d4c4-4878-4467-b936-060ee2bcca83","Type":"ContainerStarted","Data":"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014"}
Oct 30 22:09:10.855073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:10.855036    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b127d4c4-4878-4467-b936-060ee2bcca83","Type":"ContainerStarted","Data":"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1"}
Oct 30 22:09:10.855073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:10.855069    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b127d4c4-4878-4467-b936-060ee2bcca83","Type":"ContainerStarted","Data":"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8"}
Oct 30 22:09:11.160298 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:11.160222    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"
Oct 30 22:09:11.163863 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:11.163836    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pvxcb"
Oct 30 22:09:11.577732 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:11.577699    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:09:11.898671 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:11.898587    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-q4cnn"
Oct 30 22:09:12.451332 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:12.451307    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:12.995051 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:12.995025    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:12.995400 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:12.995230    5896 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 22:09:13.200974 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:13.200946    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:09:14.270632 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:14.270603    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcjjm"
Oct 30 22:09:22.154008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:22.153941    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:26.435198 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:26.435164    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-f4lgs"
Oct 30 22:09:30.644059 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:30.644024    5896 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:30.647878 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:30.647855    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7474bd8d9b-c28kq"
Oct 30 22:09:32.628168 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:09:32.628132    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:09:43.203687 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:09:43.203653    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-s5qdj"
Oct 30 22:10:02.625472 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:10:02.625435    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:10:06.043193 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:06.043164    5896 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:06.062357 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:06.062330    5896 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:06.080209 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:06.080186    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:19.826045 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:19.826011    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:10:19.826516 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:19.826425    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="alertmanager" containerID="cri-o://7ea4cac1ffe0addb15b08f364f4bb5ab401843064bb659f3dca4f4f88275903f" gracePeriod=120
Oct 30 22:10:19.826516 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:19.826477    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="kube-rbac-proxy" containerID="cri-o://6781b6106e353b46286ed2454fd1c5557241cea3df6e9f1b94097c46f7d47d2c" gracePeriod=120
Oct 30 22:10:19.826672 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:19.826483    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="prom-label-proxy" containerID="cri-o://2975e488f6a5c9a0dcde3f62fa924cb948ba10db9d0c4c038c4d5fcd1da07037" gracePeriod=120
Oct 30 22:10:19.826672 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:19.826535    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="kube-rbac-proxy-metric" containerID="cri-o://8bf0e308cab635020adee75a7be6ea9ff4f86b20263cb1c42a517e4d617f85cf" gracePeriod=120
Oct 30 22:10:19.826672 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:19.826495    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="kube-rbac-proxy-web" containerID="cri-o://593361d4247c9f5160d8d4d2d8f88dead266809da0d7ec87bac9ab79dafec66e" gracePeriod=120
Oct 30 22:10:19.826672 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:19.826554    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="config-reloader" containerID="cri-o://4aebbdb74d2638b4f2f7b9ac5c6916bcaf40ac54d094a63c6767f329769111cf" gracePeriod=120
Oct 30 22:10:20.123567 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:20.123498    5896 generic.go:356] "Generic (PLEG): container finished" podID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerID="2975e488f6a5c9a0dcde3f62fa924cb948ba10db9d0c4c038c4d5fcd1da07037" exitCode=0
Oct 30 22:10:20.123567 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:20.123520    5896 generic.go:356] "Generic (PLEG): container finished" podID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerID="6781b6106e353b46286ed2454fd1c5557241cea3df6e9f1b94097c46f7d47d2c" exitCode=0
Oct 30 22:10:20.123567 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:20.123527    5896 generic.go:356] "Generic (PLEG): container finished" podID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerID="593361d4247c9f5160d8d4d2d8f88dead266809da0d7ec87bac9ab79dafec66e" exitCode=0
Oct 30 22:10:20.123567 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:20.123533    5896 generic.go:356] "Generic (PLEG): container finished" podID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerID="4aebbdb74d2638b4f2f7b9ac5c6916bcaf40ac54d094a63c6767f329769111cf" exitCode=0
Oct 30 22:10:20.123567 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:20.123540    5896 generic.go:356] "Generic (PLEG): container finished" podID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerID="7ea4cac1ffe0addb15b08f364f4bb5ab401843064bb659f3dca4f4f88275903f" exitCode=0
Oct 30 22:10:20.123851 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:20.123571    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa31988-78b4-4a78-bac0-8dd6ae099bc5","Type":"ContainerDied","Data":"2975e488f6a5c9a0dcde3f62fa924cb948ba10db9d0c4c038c4d5fcd1da07037"}
Oct 30 22:10:20.123851 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:20.123632    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa31988-78b4-4a78-bac0-8dd6ae099bc5","Type":"ContainerDied","Data":"6781b6106e353b46286ed2454fd1c5557241cea3df6e9f1b94097c46f7d47d2c"}
Oct 30 22:10:20.123851 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:20.123676    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa31988-78b4-4a78-bac0-8dd6ae099bc5","Type":"ContainerDied","Data":"593361d4247c9f5160d8d4d2d8f88dead266809da0d7ec87bac9ab79dafec66e"}
Oct 30 22:10:20.123851 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:20.123702    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa31988-78b4-4a78-bac0-8dd6ae099bc5","Type":"ContainerDied","Data":"4aebbdb74d2638b4f2f7b9ac5c6916bcaf40ac54d094a63c6767f329769111cf"}
Oct 30 22:10:20.123851 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:20.123725    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa31988-78b4-4a78-bac0-8dd6ae099bc5","Type":"ContainerDied","Data":"7ea4cac1ffe0addb15b08f364f4bb5ab401843064bb659f3dca4f4f88275903f"}
Oct 30 22:10:21.130977 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.130947    5896 generic.go:356] "Generic (PLEG): container finished" podID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerID="8bf0e308cab635020adee75a7be6ea9ff4f86b20263cb1c42a517e4d617f85cf" exitCode=0
Oct 30 22:10:21.131308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.131017    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa31988-78b4-4a78-bac0-8dd6ae099bc5","Type":"ContainerDied","Data":"8bf0e308cab635020adee75a7be6ea9ff4f86b20263cb1c42a517e4d617f85cf"}
Oct 30 22:10:21.180512 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.180492    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:21.352984 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.352958    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-web-config\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-web-config\") pod \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") "
Oct 30 22:10:21.353126 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.353019    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-out\" (UniqueName: \"kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-out\") pod \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") "
Oct 30 22:10:21.353126 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.353071    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-volume\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-volume\") pod \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") "
Oct 30 22:10:21.353126 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.353098    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-tls-assets\" (UniqueName: \"kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-tls-assets\") pod \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") "
Oct 30 22:10:21.353284 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.353138    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy\") pod \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") "
Oct 30 22:10:21.353284 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.353171    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-main-db\") pod \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") "
Oct 30 22:10:21.353284 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.353211    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-cluster-tls-config\") pod \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") "
Oct 30 22:10:21.353284 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.353251    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-web\") pod \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") "
Oct 30 22:10:21.353493 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.353291    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-trusted-ca-bundle\") pod \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") "
Oct 30 22:10:21.353493 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.353328    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-metrics-client-ca\") pod \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") "
Oct 30 22:10:21.353493 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.353376    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-main-tls\") pod \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") "
Oct 30 22:10:21.353493 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.353404    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") "
Oct 30 22:10:21.353493 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.353451    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-kube-api-access-l7kdn\" (UniqueName: \"kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-kube-api-access-l7kdn\") pod \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\" (UID: \"0aa31988-78b4-4a78-bac0-8dd6ae099bc5\") "
Oct 30 22:10:21.353979 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.353757    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-trusted-ca-bundle" pod "0aa31988-78b4-4a78-bac0-8dd6ae099bc5" (UID: "0aa31988-78b4-4a78-bac0-8dd6ae099bc5"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:21.354508 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.354249    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-main-db" pod "0aa31988-78b4-4a78-bac0-8dd6ae099bc5" (UID: "0aa31988-78b4-4a78-bac0-8dd6ae099bc5"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:21.356106 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.356072    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-metrics-client-ca" pod "0aa31988-78b4-4a78-bac0-8dd6ae099bc5" (UID: "0aa31988-78b4-4a78-bac0-8dd6ae099bc5"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:21.356501 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.356449    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy" pod "0aa31988-78b4-4a78-bac0-8dd6ae099bc5" (UID: "0aa31988-78b4-4a78-bac0-8dd6ae099bc5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:21.356678 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.356625    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-out" pod "0aa31988-78b4-4a78-bac0-8dd6ae099bc5" (UID: "0aa31988-78b4-4a78-bac0-8dd6ae099bc5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:21.356782 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.356737    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-volume" pod "0aa31988-78b4-4a78-bac0-8dd6ae099bc5" (UID: "0aa31988-78b4-4a78-bac0-8dd6ae099bc5"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:21.356782 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.356755    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-tls-assets" pod "0aa31988-78b4-4a78-bac0-8dd6ae099bc5" (UID: "0aa31988-78b4-4a78-bac0-8dd6ae099bc5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:21.357260 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.357240    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-kube-api-access-l7kdn" pod "0aa31988-78b4-4a78-bac0-8dd6ae099bc5" (UID: "0aa31988-78b4-4a78-bac0-8dd6ae099bc5"). InnerVolumeSpecName "kube-api-access-l7kdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:21.358008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.357989    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-main-tls" pod "0aa31988-78b4-4a78-bac0-8dd6ae099bc5" (UID: "0aa31988-78b4-4a78-bac0-8dd6ae099bc5"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:21.358200 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.358179    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-web" pod "0aa31988-78b4-4a78-bac0-8dd6ae099bc5" (UID: "0aa31988-78b4-4a78-bac0-8dd6ae099bc5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:21.358531 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.358512    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-metric" pod "0aa31988-78b4-4a78-bac0-8dd6ae099bc5" (UID: "0aa31988-78b4-4a78-bac0-8dd6ae099bc5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:21.361522 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.361367    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-cluster-tls-config" pod "0aa31988-78b4-4a78-bac0-8dd6ae099bc5" (UID: "0aa31988-78b4-4a78-bac0-8dd6ae099bc5"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:21.368940 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.368858    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-web-config" pod "0aa31988-78b4-4a78-bac0-8dd6ae099bc5" (UID: "0aa31988-78b4-4a78-bac0-8dd6ae099bc5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:21.454307 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.454285    5896 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-volume\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:21.454307 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.454306    5896 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-tls-assets\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:21.454421 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.454316    5896 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:21.454421 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.454327    5896 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-main-db\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:21.454421 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.454338    5896 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-cluster-tls-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:21.454421 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.454347    5896 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:21.454421 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.454356    5896 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:21.454421 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.454364    5896 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-metrics-client-ca\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:21.454421 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.454374    5896 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-main-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:21.454421 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.454382    5896 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:21.454421 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.454391    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7kdn\" (UniqueName: \"kubernetes.io/projected/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-kube-api-access-l7kdn\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:21.454421 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.454401    5896 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-web-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:21.454421 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:21.454408    5896 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0aa31988-78b4-4a78-bac0-8dd6ae099bc5-config-out\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:22.138953 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.138879    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0aa31988-78b4-4a78-bac0-8dd6ae099bc5","Type":"ContainerDied","Data":"52876bb9d9c8bfdd32770534e2826f94a7495a77322575d113bd25cafd7088b9"}
Oct 30 22:10:22.138953 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.138923    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.139443 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.138985    5896 scope.go:122] "RemoveContainer" containerID="2975e488f6a5c9a0dcde3f62fa924cb948ba10db9d0c4c038c4d5fcd1da07037"
Oct 30 22:10:22.146895 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.146875    5896 scope.go:122] "RemoveContainer" containerID="8bf0e308cab635020adee75a7be6ea9ff4f86b20263cb1c42a517e4d617f85cf"
Oct 30 22:10:22.153668 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.153622    5896 scope.go:122] "RemoveContainer" containerID="6781b6106e353b46286ed2454fd1c5557241cea3df6e9f1b94097c46f7d47d2c"
Oct 30 22:10:22.160497 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.160477    5896 scope.go:122] "RemoveContainer" containerID="593361d4247c9f5160d8d4d2d8f88dead266809da0d7ec87bac9ab79dafec66e"
Oct 30 22:10:22.166940 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.166922    5896 scope.go:122] "RemoveContainer" containerID="4aebbdb74d2638b4f2f7b9ac5c6916bcaf40ac54d094a63c6767f329769111cf"
Oct 30 22:10:22.173521 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.173505    5896 scope.go:122] "RemoveContainer" containerID="7ea4cac1ffe0addb15b08f364f4bb5ab401843064bb659f3dca4f4f88275903f"
Oct 30 22:10:22.180332 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.180316    5896 scope.go:122] "RemoveContainer" containerID="8d07db46466466c90462241b0d4bee2de008cd750736078bcec8f5f26c371470"
Oct 30 22:10:22.363561 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.363536    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:10:22.414306 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.414227    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:10:22.414688 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.414670    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:10:22.414958 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.414946    5896 cpu_manager.go:414] "containerMap: removing container" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="kube-rbac-proxy-metric"
Oct 30 22:10:22.414994 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.414960    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="kube-rbac-proxy-metric"
Oct 30 22:10:22.414994 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.414971    5896 cpu_manager.go:414] "containerMap: removing container" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="config-reloader"
Oct 30 22:10:22.414994 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.414976    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="config-reloader"
Oct 30 22:10:22.414994 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.414990    5896 cpu_manager.go:414] "containerMap: removing container" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="kube-rbac-proxy"
Oct 30 22:10:22.414994 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.414994    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="kube-rbac-proxy"
Oct 30 22:10:22.415182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.415007    5896 cpu_manager.go:414] "containerMap: removing container" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="kube-rbac-proxy-web"
Oct 30 22:10:22.415182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.415012    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="kube-rbac-proxy-web"
Oct 30 22:10:22.415182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.415023    5896 cpu_manager.go:414] "containerMap: removing container" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="init-config-reloader"
Oct 30 22:10:22.415182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.415027    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="init-config-reloader"
Oct 30 22:10:22.415182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.415039    5896 cpu_manager.go:414] "containerMap: removing container" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="prom-label-proxy"
Oct 30 22:10:22.415182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.415043    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="prom-label-proxy"
Oct 30 22:10:22.415182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.415052    5896 cpu_manager.go:414] "containerMap: removing container" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="alertmanager"
Oct 30 22:10:22.415182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.415056    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="alertmanager"
Oct 30 22:10:22.415182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.415100    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="init-config-reloader"
Oct 30 22:10:22.415182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.415109    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="prom-label-proxy"
Oct 30 22:10:22.415182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.415115    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="alertmanager"
Oct 30 22:10:22.415182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.415120    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="kube-rbac-proxy-metric"
Oct 30 22:10:22.415182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.415126    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="config-reloader"
Oct 30 22:10:22.415182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.415132    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="kube-rbac-proxy"
Oct 30 22:10:22.415182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.415138    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" containerName="kube-rbac-proxy-web"
Oct 30 22:10:22.418438 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.418423    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.463146 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.463117    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a15ec80d-1d69-4956-b781-03a41fa68f7d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.463146 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.463150    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.463329 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.463176    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.463329 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.463280    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmkbc\" (UniqueName: \"kubernetes.io/projected/a15ec80d-1d69-4956-b781-03a41fa68f7d-kube-api-access-bmkbc\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.463329 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.463299    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15ec80d-1d69-4956-b781-03a41fa68f7d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.463329 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.463317    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.463449 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.463343    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a15ec80d-1d69-4956-b781-03a41fa68f7d-config-out\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.463449 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.463381    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a15ec80d-1d69-4956-b781-03a41fa68f7d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.463449 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.463422    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.463449 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.463443    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-config-volume\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.463566 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.463458    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.463566 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.463482    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a15ec80d-1d69-4956-b781-03a41fa68f7d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.463566 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.463551    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-web-config\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.473317 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.473285    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 22:10:22.473455 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.473366    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 22:10:22.473523 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.473487    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 22:10:22.473618 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.473592    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 22:10:22.473618 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.473618    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 22:10:22.473815 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.473621    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 22:10:22.473815 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.473657    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 22:10:22.473815 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.473759    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tc949\""
Oct 30 22:10:22.473815 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.473790    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 22:10:22.524032 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.524006    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 22:10:22.564060 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.564037    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-config-volume\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.564220 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.564073    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.564283 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.564252    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a15ec80d-1d69-4956-b781-03a41fa68f7d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.564336 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.564288    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-web-config\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.564336 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.564322    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a15ec80d-1d69-4956-b781-03a41fa68f7d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.564443 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.564351    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.564443 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.564396    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.564539 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.564444    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmkbc\" (UniqueName: \"kubernetes.io/projected/a15ec80d-1d69-4956-b781-03a41fa68f7d-kube-api-access-bmkbc\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.564539 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.564470    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15ec80d-1d69-4956-b781-03a41fa68f7d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.564539 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.564471    5896 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 22:10:22.564539 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.564499    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.564539 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.564532    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a15ec80d-1d69-4956-b781-03a41fa68f7d-config-out\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.564806 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.564557    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a15ec80d-1d69-4956-b781-03a41fa68f7d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.564806 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.564582    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.564806 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.564776    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a15ec80d-1d69-4956-b781-03a41fa68f7d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.565791 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.565739    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a15ec80d-1d69-4956-b781-03a41fa68f7d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.566306 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.566276    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a15ec80d-1d69-4956-b781-03a41fa68f7d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.567586 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.567558    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a15ec80d-1d69-4956-b781-03a41fa68f7d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.567797 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.567769    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.567923 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.567882    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-config-volume\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.568565 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.568520    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-web-config\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.568690 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.568593    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a15ec80d-1d69-4956-b781-03a41fa68f7d-config-out\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.568749 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.568704    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.568792 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.568748    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.569077 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.569056    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.569443 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.569423    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a15ec80d-1d69-4956-b781-03a41fa68f7d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.627790 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.627764    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmkbc\" (UniqueName: \"kubernetes.io/projected/a15ec80d-1d69-4956-b781-03a41fa68f7d-kube-api-access-bmkbc\") pod \"alertmanager-main-0\" (UID: \"a15ec80d-1d69-4956-b781-03a41fa68f7d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:22.665319 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.665256    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:10:22.734240 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:22.734207    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:10:23.173919 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:23.173530    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:10:23.176673 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:10:23.176621    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda15ec80d_1d69_4956_b781_03a41fa68f7d.slice/crio-f0a4d47d11d3891bed35313b1316ee0b6fad7f296912e2b08a5486527752961f WatchSource:0}: Error finding container f0a4d47d11d3891bed35313b1316ee0b6fad7f296912e2b08a5486527752961f: Status 404 returned error can't find the container with id f0a4d47d11d3891bed35313b1316ee0b6fad7f296912e2b08a5486527752961f
Oct 30 22:10:23.735340 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:23.735309    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0aa31988-78b4-4a78-bac0-8dd6ae099bc5" path="/var/lib/kubelet/pods/0aa31988-78b4-4a78-bac0-8dd6ae099bc5/volumes"
Oct 30 22:10:24.148450 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:24.148368    5896 generic.go:356] "Generic (PLEG): container finished" podID="a15ec80d-1d69-4956-b781-03a41fa68f7d" containerID="80a0d8b56b41e68d8367d83bb6cbf145e3e71b649d671f9f57f707eb4ce0fb39" exitCode=0
Oct 30 22:10:24.148450 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:24.148416    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a15ec80d-1d69-4956-b781-03a41fa68f7d","Type":"ContainerDied","Data":"80a0d8b56b41e68d8367d83bb6cbf145e3e71b649d671f9f57f707eb4ce0fb39"}
Oct 30 22:10:24.148707 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:24.148469    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a15ec80d-1d69-4956-b781-03a41fa68f7d","Type":"ContainerStarted","Data":"f0a4d47d11d3891bed35313b1316ee0b6fad7f296912e2b08a5486527752961f"}
Oct 30 22:10:25.157302 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.157264    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a15ec80d-1d69-4956-b781-03a41fa68f7d","Type":"ContainerStarted","Data":"12ff7f0d3de3e6ce534841284987731ebe1a87adbf1439794836f3237309cf74"}
Oct 30 22:10:25.157738 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.157338    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a15ec80d-1d69-4956-b781-03a41fa68f7d","Type":"ContainerStarted","Data":"f3a21bc478528d19e76bcf8ed4f5359edfeea191f0b8ab0bf4331a54855b2587"}
Oct 30 22:10:25.157738 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.157374    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a15ec80d-1d69-4956-b781-03a41fa68f7d","Type":"ContainerStarted","Data":"78217a43a9890f2b73c69f8672ff3e95d00b7e58edfaba8af75b4d37f524cf30"}
Oct 30 22:10:25.157738 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.157399    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a15ec80d-1d69-4956-b781-03a41fa68f7d","Type":"ContainerStarted","Data":"340c6d251e7c47518f42625df16e0befd4f6621cf7ae6b325054ba5f8d02b6e7"}
Oct 30 22:10:25.157738 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.157426    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a15ec80d-1d69-4956-b781-03a41fa68f7d","Type":"ContainerStarted","Data":"e9ab5bc2b665a067e386bbec06a01651123a846ed8994fcd169d4ceec993275f"}
Oct 30 22:10:25.157738 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.157463    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a15ec80d-1d69-4956-b781-03a41fa68f7d","Type":"ContainerStarted","Data":"24233d1cba01e64a6420bfd154033af48ffb73348d035225cb0a4b161749e7a7"}
Oct 30 22:10:25.290935 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.290875    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.279223913 podStartE2EDuration="3.290859748s" totalImagesPullingTime="0s" totalInitContainerRuntime="11.635835ms" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:25.290397214 +0000 UTC m=+77.701863181" watchObservedRunningTime="2026-10-30 22:10:25.290859748 +0000 UTC m=+77.702325716"
Oct 30 22:10:25.645177 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.645145    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:10:25.645455 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.645415    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="prometheus" containerID="cri-o://54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8" gracePeriod=600
Oct 30 22:10:25.645596 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.645451    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="thanos-sidecar" containerID="cri-o://c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014" gracePeriod=600
Oct 30 22:10:25.645596 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.645451    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="kube-rbac-proxy" containerID="cri-o://f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d" gracePeriod=600
Oct 30 22:10:25.645596 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.645483    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="config-reloader" containerID="cri-o://454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1" gracePeriod=600
Oct 30 22:10:25.645596 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.645510    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="kube-rbac-proxy-web" containerID="cri-o://754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916" gracePeriod=600
Oct 30 22:10:25.645850 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.645668    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="kube-rbac-proxy-thanos" containerID="cri-o://52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525" gracePeriod=600
Oct 30 22:10:25.899934 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.899878    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:25.994840 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.994817    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-trusted-ca-bundle\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.994986 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.994850    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-1\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.994986 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.994879    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-kubelet-serving-ca-bundle\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.994986 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.994916    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-thanos-prometheus-http-client-file\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.994986 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.994961    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-config-out\" (UniqueName: \"kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-config-out\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.995181 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.994993    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-0\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.995181 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995029    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-kube-rbac-proxy\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.995181 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995063    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-db\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.995181 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995090    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-kube-api-access-bh5mn\" (UniqueName: \"kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-kube-api-access-bh5mn\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.995181 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995128    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-tls\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.995181 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995161    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-2\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.995456 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995193    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-tls-assets\" (UniqueName: \"kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-tls-assets\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.995456 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995227    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-grpc-tls\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.995456 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995269    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-metrics-client-certs\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.995456 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995213    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-1" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:25.995456 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995269    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-trusted-ca-bundle" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:25.995723 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995487    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.995723 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995538    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.995723 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995570    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-serving-certs-ca-bundle\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.995723 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995623    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-kubelet-serving-ca-bundle" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:25.995926 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995751    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-metrics-client-ca\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.995926 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995834    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-config\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-config\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.995926 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.995879    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-web-config\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-web-config\") pod \"b127d4c4-4878-4467-b936-060ee2bcca83\" (UID: \"b127d4c4-4878-4467-b936-060ee2bcca83\") "
Oct 30 22:10:25.996176 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.996154    5896 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:25.996231 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.996183    5896 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:25.996231 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.996200    5896 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:25.996660 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.996608    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-metrics-client-ca" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:25.996775 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.996660    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-serving-certs-ca-bundle" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:25.996917 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.996892    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-2" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:25.997564 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.997312    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-0" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:25.998092 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.998054    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-db" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:25.998769 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.998744    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-config-out" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:25.999116 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:25.999064    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-kube-rbac-proxy" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:26.000126 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.000086    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-thanos-prometheus-http-client-file" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:26.000315 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.000291    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-metrics-client-certs" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:26.000885 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.000798    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-kube-api-access-bh5mn" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "kube-api-access-bh5mn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:26.000885 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.000851    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-thanos-sidecar-tls" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:26.001130 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.001106    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-tls-assets" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:26.001185 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.001120    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-config" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:26.001219 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.001197    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-grpc-tls" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:26.001378 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.001360    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-kube-rbac-proxy-web" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:26.001824 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.001805    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-tls" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:26.011471 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.011453    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-web-config" pod "b127d4c4-4878-4467-b936-060ee2bcca83" (UID: "b127d4c4-4878-4467-b936-060ee2bcca83"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:26.097375 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097345    5896 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.097375 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097371    5896 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-config-out\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.097537 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097381    5896 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.097537 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097392    5896 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-kube-rbac-proxy\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.097537 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097401    5896 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-db\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.097537 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097410    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh5mn\" (UniqueName: \"kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-kube-api-access-bh5mn\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.097537 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097420    5896 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.097537 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097429    5896 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.097537 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097437    5896 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b127d4c4-4878-4467-b936-060ee2bcca83-tls-assets\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.097537 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097445    5896 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-grpc-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.097537 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097453    5896 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-metrics-client-certs\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.097537 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097462    5896 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.097537 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097471    5896 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.097537 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097480    5896 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.097537 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097488    5896 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b127d4c4-4878-4467-b936-060ee2bcca83-configmap-metrics-client-ca\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.097537 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097497    5896 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.097537 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.097505    5896 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b127d4c4-4878-4467-b936-060ee2bcca83-web-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:26.164491 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.164425    5896 generic.go:356] "Generic (PLEG): container finished" podID="b127d4c4-4878-4467-b936-060ee2bcca83" containerID="52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525" exitCode=0
Oct 30 22:10:26.164491 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.164448    5896 generic.go:356] "Generic (PLEG): container finished" podID="b127d4c4-4878-4467-b936-060ee2bcca83" containerID="f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d" exitCode=0
Oct 30 22:10:26.164491 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.164456    5896 generic.go:356] "Generic (PLEG): container finished" podID="b127d4c4-4878-4467-b936-060ee2bcca83" containerID="754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916" exitCode=0
Oct 30 22:10:26.164491 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.164467    5896 generic.go:356] "Generic (PLEG): container finished" podID="b127d4c4-4878-4467-b936-060ee2bcca83" containerID="c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014" exitCode=0
Oct 30 22:10:26.164491 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.164477    5896 generic.go:356] "Generic (PLEG): container finished" podID="b127d4c4-4878-4467-b936-060ee2bcca83" containerID="454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1" exitCode=0
Oct 30 22:10:26.164491 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.164482    5896 generic.go:356] "Generic (PLEG): container finished" podID="b127d4c4-4878-4467-b936-060ee2bcca83" containerID="54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8" exitCode=0
Oct 30 22:10:26.165096 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.164502    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.165096 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.164516    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b127d4c4-4878-4467-b936-060ee2bcca83","Type":"ContainerDied","Data":"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525"}
Oct 30 22:10:26.165096 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.164601    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b127d4c4-4878-4467-b936-060ee2bcca83","Type":"ContainerDied","Data":"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d"}
Oct 30 22:10:26.165096 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.164673    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b127d4c4-4878-4467-b936-060ee2bcca83","Type":"ContainerDied","Data":"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916"}
Oct 30 22:10:26.165096 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.164702    5896 scope.go:122] "RemoveContainer" containerID="52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525"
Oct 30 22:10:26.165096 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.164716    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b127d4c4-4878-4467-b936-060ee2bcca83","Type":"ContainerDied","Data":"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014"}
Oct 30 22:10:26.165096 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.164756    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b127d4c4-4878-4467-b936-060ee2bcca83","Type":"ContainerDied","Data":"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1"}
Oct 30 22:10:26.165096 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.164797    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b127d4c4-4878-4467-b936-060ee2bcca83","Type":"ContainerDied","Data":"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8"}
Oct 30 22:10:26.165096 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.164837    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b127d4c4-4878-4467-b936-060ee2bcca83","Type":"ContainerDied","Data":"7c1405656695dd347a4c305286ea14ad41d060a3efd2a1f7ce26df51bb4f26d1"}
Oct 30 22:10:26.173954 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.173932    5896 scope.go:122] "RemoveContainer" containerID="f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d"
Oct 30 22:10:26.180758 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.180733    5896 scope.go:122] "RemoveContainer" containerID="754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916"
Oct 30 22:10:26.188000 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.187986    5896 scope.go:122] "RemoveContainer" containerID="c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014"
Oct 30 22:10:26.194409 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.194395    5896 scope.go:122] "RemoveContainer" containerID="454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1"
Oct 30 22:10:26.201051 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.201035    5896 scope.go:122] "RemoveContainer" containerID="54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8"
Oct 30 22:10:26.207991 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.207886    5896 scope.go:122] "RemoveContainer" containerID="595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe"
Oct 30 22:10:26.214114 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.214101    5896 scope.go:122] "RemoveContainer" containerID="52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525"
Oct 30 22:10:26.214354 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:10:26.214336    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525\": container with ID starting with 52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525 not found: ID does not exist" containerID="52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525"
Oct 30 22:10:26.214404 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.214364    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525"} err="failed to get container status \"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525\": rpc error: code = NotFound desc = could not find container \"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525\": container with ID starting with 52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525 not found: ID does not exist"
Oct 30 22:10:26.214404 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.214393    5896 scope.go:122] "RemoveContainer" containerID="f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d"
Oct 30 22:10:26.214612 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:10:26.214596    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d\": container with ID starting with f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d not found: ID does not exist" containerID="f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d"
Oct 30 22:10:26.214666 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.214615    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d"} err="failed to get container status \"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d\": rpc error: code = NotFound desc = could not find container \"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d\": container with ID starting with f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d not found: ID does not exist"
Oct 30 22:10:26.214666 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.214628    5896 scope.go:122] "RemoveContainer" containerID="754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916"
Oct 30 22:10:26.214874 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:10:26.214857    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916\": container with ID starting with 754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916 not found: ID does not exist" containerID="754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916"
Oct 30 22:10:26.214917 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.214879    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916"} err="failed to get container status \"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916\": rpc error: code = NotFound desc = could not find container \"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916\": container with ID starting with 754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916 not found: ID does not exist"
Oct 30 22:10:26.214917 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.214893    5896 scope.go:122] "RemoveContainer" containerID="c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014"
Oct 30 22:10:26.215099 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:10:26.215083    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014\": container with ID starting with c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014 not found: ID does not exist" containerID="c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014"
Oct 30 22:10:26.215139 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.215103    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014"} err="failed to get container status \"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014\": rpc error: code = NotFound desc = could not find container \"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014\": container with ID starting with c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014 not found: ID does not exist"
Oct 30 22:10:26.215139 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.215116    5896 scope.go:122] "RemoveContainer" containerID="454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1"
Oct 30 22:10:26.215319 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:10:26.215305    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1\": container with ID starting with 454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1 not found: ID does not exist" containerID="454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1"
Oct 30 22:10:26.215362 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.215322    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1"} err="failed to get container status \"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1\": rpc error: code = NotFound desc = could not find container \"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1\": container with ID starting with 454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1 not found: ID does not exist"
Oct 30 22:10:26.215362 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.215333    5896 scope.go:122] "RemoveContainer" containerID="54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8"
Oct 30 22:10:26.215520 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:10:26.215502    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8\": container with ID starting with 54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8 not found: ID does not exist" containerID="54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8"
Oct 30 22:10:26.215552 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.215522    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8"} err="failed to get container status \"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8\": rpc error: code = NotFound desc = could not find container \"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8\": container with ID starting with 54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8 not found: ID does not exist"
Oct 30 22:10:26.215552 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.215533    5896 scope.go:122] "RemoveContainer" containerID="595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe"
Oct 30 22:10:26.215775 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:10:26.215755    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe\": container with ID starting with 595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe not found: ID does not exist" containerID="595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe"
Oct 30 22:10:26.215829 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.215778    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe"} err="failed to get container status \"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe\": rpc error: code = NotFound desc = could not find container \"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe\": container with ID starting with 595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe not found: ID does not exist"
Oct 30 22:10:26.215829 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.215788    5896 scope.go:122] "RemoveContainer" containerID="52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525"
Oct 30 22:10:26.216046 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.216029    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525"} err="failed to get container status \"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525\": rpc error: code = NotFound desc = could not find container \"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525\": container with ID starting with 52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525 not found: ID does not exist"
Oct 30 22:10:26.216098 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.216047    5896 scope.go:122] "RemoveContainer" containerID="f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d"
Oct 30 22:10:26.216314 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.216296    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d"} err="failed to get container status \"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d\": rpc error: code = NotFound desc = could not find container \"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d\": container with ID starting with f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d not found: ID does not exist"
Oct 30 22:10:26.216379 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.216316    5896 scope.go:122] "RemoveContainer" containerID="754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916"
Oct 30 22:10:26.216551 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.216534    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916"} err="failed to get container status \"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916\": rpc error: code = NotFound desc = could not find container \"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916\": container with ID starting with 754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916 not found: ID does not exist"
Oct 30 22:10:26.216591 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.216551    5896 scope.go:122] "RemoveContainer" containerID="c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014"
Oct 30 22:10:26.216788 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.216769    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014"} err="failed to get container status \"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014\": rpc error: code = NotFound desc = could not find container \"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014\": container with ID starting with c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014 not found: ID does not exist"
Oct 30 22:10:26.216863 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.216789    5896 scope.go:122] "RemoveContainer" containerID="454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1"
Oct 30 22:10:26.217007 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.216991    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1"} err="failed to get container status \"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1\": rpc error: code = NotFound desc = could not find container \"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1\": container with ID starting with 454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1 not found: ID does not exist"
Oct 30 22:10:26.217052 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.217009    5896 scope.go:122] "RemoveContainer" containerID="54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8"
Oct 30 22:10:26.217209 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.217192    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8"} err="failed to get container status \"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8\": rpc error: code = NotFound desc = could not find container \"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8\": container with ID starting with 54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8 not found: ID does not exist"
Oct 30 22:10:26.217275 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.217210    5896 scope.go:122] "RemoveContainer" containerID="595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe"
Oct 30 22:10:26.217409 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.217394    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe"} err="failed to get container status \"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe\": rpc error: code = NotFound desc = could not find container \"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe\": container with ID starting with 595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe not found: ID does not exist"
Oct 30 22:10:26.217467 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.217410    5896 scope.go:122] "RemoveContainer" containerID="52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525"
Oct 30 22:10:26.217590 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.217575    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525"} err="failed to get container status \"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525\": rpc error: code = NotFound desc = could not find container \"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525\": container with ID starting with 52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525 not found: ID does not exist"
Oct 30 22:10:26.217633 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.217590    5896 scope.go:122] "RemoveContainer" containerID="f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d"
Oct 30 22:10:26.217778 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.217761    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d"} err="failed to get container status \"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d\": rpc error: code = NotFound desc = could not find container \"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d\": container with ID starting with f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d not found: ID does not exist"
Oct 30 22:10:26.217844 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.217779    5896 scope.go:122] "RemoveContainer" containerID="754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916"
Oct 30 22:10:26.218001 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.217985    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916"} err="failed to get container status \"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916\": rpc error: code = NotFound desc = could not find container \"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916\": container with ID starting with 754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916 not found: ID does not exist"
Oct 30 22:10:26.218047 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.218000    5896 scope.go:122] "RemoveContainer" containerID="c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014"
Oct 30 22:10:26.218202 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.218183    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014"} err="failed to get container status \"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014\": rpc error: code = NotFound desc = could not find container \"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014\": container with ID starting with c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014 not found: ID does not exist"
Oct 30 22:10:26.218268 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.218202    5896 scope.go:122] "RemoveContainer" containerID="454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1"
Oct 30 22:10:26.218397 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.218383    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1"} err="failed to get container status \"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1\": rpc error: code = NotFound desc = could not find container \"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1\": container with ID starting with 454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1 not found: ID does not exist"
Oct 30 22:10:26.218439 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.218397    5896 scope.go:122] "RemoveContainer" containerID="54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8"
Oct 30 22:10:26.218630 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.218606    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8"} err="failed to get container status \"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8\": rpc error: code = NotFound desc = could not find container \"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8\": container with ID starting with 54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8 not found: ID does not exist"
Oct 30 22:10:26.218697 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.218632    5896 scope.go:122] "RemoveContainer" containerID="595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe"
Oct 30 22:10:26.218865 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.218849    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe"} err="failed to get container status \"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe\": rpc error: code = NotFound desc = could not find container \"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe\": container with ID starting with 595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe not found: ID does not exist"
Oct 30 22:10:26.218904 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.218865    5896 scope.go:122] "RemoveContainer" containerID="52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525"
Oct 30 22:10:26.219070 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.219052    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525"} err="failed to get container status \"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525\": rpc error: code = NotFound desc = could not find container \"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525\": container with ID starting with 52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525 not found: ID does not exist"
Oct 30 22:10:26.219136 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.219070    5896 scope.go:122] "RemoveContainer" containerID="f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d"
Oct 30 22:10:26.219337 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.219316    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d"} err="failed to get container status \"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d\": rpc error: code = NotFound desc = could not find container \"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d\": container with ID starting with f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d not found: ID does not exist"
Oct 30 22:10:26.219337 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.219336    5896 scope.go:122] "RemoveContainer" containerID="754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916"
Oct 30 22:10:26.219519 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.219501    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916"} err="failed to get container status \"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916\": rpc error: code = NotFound desc = could not find container \"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916\": container with ID starting with 754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916 not found: ID does not exist"
Oct 30 22:10:26.219575 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.219521    5896 scope.go:122] "RemoveContainer" containerID="c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014"
Oct 30 22:10:26.219759 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.219742    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014"} err="failed to get container status \"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014\": rpc error: code = NotFound desc = could not find container \"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014\": container with ID starting with c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014 not found: ID does not exist"
Oct 30 22:10:26.219759 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.219758    5896 scope.go:122] "RemoveContainer" containerID="454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1"
Oct 30 22:10:26.219934 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.219918    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1"} err="failed to get container status \"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1\": rpc error: code = NotFound desc = could not find container \"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1\": container with ID starting with 454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1 not found: ID does not exist"
Oct 30 22:10:26.219934 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.219932    5896 scope.go:122] "RemoveContainer" containerID="54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8"
Oct 30 22:10:26.220146 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.220126    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8"} err="failed to get container status \"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8\": rpc error: code = NotFound desc = could not find container \"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8\": container with ID starting with 54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8 not found: ID does not exist"
Oct 30 22:10:26.220146 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.220145    5896 scope.go:122] "RemoveContainer" containerID="595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe"
Oct 30 22:10:26.220320 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.220303    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe"} err="failed to get container status \"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe\": rpc error: code = NotFound desc = could not find container \"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe\": container with ID starting with 595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe not found: ID does not exist"
Oct 30 22:10:26.220320 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.220318    5896 scope.go:122] "RemoveContainer" containerID="52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525"
Oct 30 22:10:26.220539 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.220523    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525"} err="failed to get container status \"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525\": rpc error: code = NotFound desc = could not find container \"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525\": container with ID starting with 52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525 not found: ID does not exist"
Oct 30 22:10:26.220539 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.220538    5896 scope.go:122] "RemoveContainer" containerID="f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d"
Oct 30 22:10:26.220782 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.220764    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d"} err="failed to get container status \"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d\": rpc error: code = NotFound desc = could not find container \"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d\": container with ID starting with f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d not found: ID does not exist"
Oct 30 22:10:26.220825 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.220783    5896 scope.go:122] "RemoveContainer" containerID="754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916"
Oct 30 22:10:26.220996 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.220980    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916"} err="failed to get container status \"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916\": rpc error: code = NotFound desc = could not find container \"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916\": container with ID starting with 754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916 not found: ID does not exist"
Oct 30 22:10:26.221035 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.220996    5896 scope.go:122] "RemoveContainer" containerID="c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014"
Oct 30 22:10:26.221232 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.221209    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014"} err="failed to get container status \"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014\": rpc error: code = NotFound desc = could not find container \"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014\": container with ID starting with c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014 not found: ID does not exist"
Oct 30 22:10:26.221232 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.221230    5896 scope.go:122] "RemoveContainer" containerID="454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1"
Oct 30 22:10:26.221437 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.221421    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1"} err="failed to get container status \"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1\": rpc error: code = NotFound desc = could not find container \"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1\": container with ID starting with 454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1 not found: ID does not exist"
Oct 30 22:10:26.221479 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.221437    5896 scope.go:122] "RemoveContainer" containerID="54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8"
Oct 30 22:10:26.221650 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.221620    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8"} err="failed to get container status \"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8\": rpc error: code = NotFound desc = could not find container \"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8\": container with ID starting with 54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8 not found: ID does not exist"
Oct 30 22:10:26.221699 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.221635    5896 scope.go:122] "RemoveContainer" containerID="595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe"
Oct 30 22:10:26.221870 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.221851    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe"} err="failed to get container status \"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe\": rpc error: code = NotFound desc = could not find container \"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe\": container with ID starting with 595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe not found: ID does not exist"
Oct 30 22:10:26.221942 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.221872    5896 scope.go:122] "RemoveContainer" containerID="52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525"
Oct 30 22:10:26.222095 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.222079    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525"} err="failed to get container status \"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525\": rpc error: code = NotFound desc = could not find container \"52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525\": container with ID starting with 52707d211722dd6054b5103c872feb3d99a3417abbef3ccdd1cb390fda1fb525 not found: ID does not exist"
Oct 30 22:10:26.222138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.222094    5896 scope.go:122] "RemoveContainer" containerID="f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d"
Oct 30 22:10:26.222309 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.222291    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d"} err="failed to get container status \"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d\": rpc error: code = NotFound desc = could not find container \"f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d\": container with ID starting with f17970cb3b79d1db7cbf3d70f0cfaab6f32509bc244ca06de96cfcf07483544d not found: ID does not exist"
Oct 30 22:10:26.222381 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.222311    5896 scope.go:122] "RemoveContainer" containerID="754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916"
Oct 30 22:10:26.222555 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.222534    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916"} err="failed to get container status \"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916\": rpc error: code = NotFound desc = could not find container \"754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916\": container with ID starting with 754e8ee744c7ebcc2e0097fdd694e7ed44d62d5a487ea66c7bc88ed7bf8b6916 not found: ID does not exist"
Oct 30 22:10:26.222598 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.222556    5896 scope.go:122] "RemoveContainer" containerID="c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014"
Oct 30 22:10:26.222764 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.222748    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014"} err="failed to get container status \"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014\": rpc error: code = NotFound desc = could not find container \"c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014\": container with ID starting with c99361828a7b0f56ed1ae5bd425b1e7e5eae175da692a6ef48d6f9fa7091b014 not found: ID does not exist"
Oct 30 22:10:26.222861 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.222764    5896 scope.go:122] "RemoveContainer" containerID="454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1"
Oct 30 22:10:26.222984 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.222966    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1"} err="failed to get container status \"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1\": rpc error: code = NotFound desc = could not find container \"454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1\": container with ID starting with 454855aefead7adb66500fca2f7b1bc7f7c61e5a90b80fd40a9ea47932f7bae1 not found: ID does not exist"
Oct 30 22:10:26.223026 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.222985    5896 scope.go:122] "RemoveContainer" containerID="54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8"
Oct 30 22:10:26.223227 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.223207    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8"} err="failed to get container status \"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8\": rpc error: code = NotFound desc = could not find container \"54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8\": container with ID starting with 54c1018bdfb1553d5a83f6528653d90f00f7d2317ccd9b599a6a433aeca816c8 not found: ID does not exist"
Oct 30 22:10:26.223277 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.223228    5896 scope.go:122] "RemoveContainer" containerID="595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe"
Oct 30 22:10:26.223437 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.223422    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe"} err="failed to get container status \"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe\": rpc error: code = NotFound desc = could not find container \"595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe\": container with ID starting with 595ecf18f6da34a037124930d803e797891785f4fa7535007faceef00a4509fe not found: ID does not exist"
Oct 30 22:10:26.338901 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.338878    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:10:26.347076 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.347054    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:10:26.381549 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.381527    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:10:26.381900 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.381888    5896 cpu_manager.go:414] "containerMap: removing container" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="prometheus"
Oct 30 22:10:26.381951 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.381903    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="prometheus"
Oct 30 22:10:26.381951 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.381917    5896 cpu_manager.go:414] "containerMap: removing container" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="kube-rbac-proxy-web"
Oct 30 22:10:26.381951 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.381922    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="kube-rbac-proxy-web"
Oct 30 22:10:26.381951 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.381947    5896 cpu_manager.go:414] "containerMap: removing container" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="config-reloader"
Oct 30 22:10:26.382073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.381953    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="config-reloader"
Oct 30 22:10:26.382073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.381959    5896 cpu_manager.go:414] "containerMap: removing container" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="thanos-sidecar"
Oct 30 22:10:26.382073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.381963    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="thanos-sidecar"
Oct 30 22:10:26.382073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.381970    5896 cpu_manager.go:414] "containerMap: removing container" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="kube-rbac-proxy"
Oct 30 22:10:26.382073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.381974    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="kube-rbac-proxy"
Oct 30 22:10:26.382073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.381980    5896 cpu_manager.go:414] "containerMap: removing container" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="init-config-reloader"
Oct 30 22:10:26.382073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.381984    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="init-config-reloader"
Oct 30 22:10:26.382073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.381993    5896 cpu_manager.go:414] "containerMap: removing container" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="kube-rbac-proxy-thanos"
Oct 30 22:10:26.382073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.381997    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="kube-rbac-proxy-thanos"
Oct 30 22:10:26.382073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.382049    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="prometheus"
Oct 30 22:10:26.382073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.382057    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="kube-rbac-proxy-web"
Oct 30 22:10:26.382073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.382065    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="config-reloader"
Oct 30 22:10:26.382073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.382070    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="thanos-sidecar"
Oct 30 22:10:26.382073 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.382075    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="kube-rbac-proxy"
Oct 30 22:10:26.382436 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.382082    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" containerName="kube-rbac-proxy-thanos"
Oct 30 22:10:26.385701 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.385689    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.400773 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.400752    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/350f2760-583f-4109-9d15-b4a82ac2d43a-config-out\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.400857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.400779    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.400857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.400797    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-config\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.400857 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.400811    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.400981 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.400869    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.401012 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.400972    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.401012 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.401002    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.401067 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.401033    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.401067 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.401052    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.401172 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.401109    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/350f2760-583f-4109-9d15-b4a82ac2d43a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.401172 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.401158    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.401271 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.401197    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.401271 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.401245    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/350f2760-583f-4109-9d15-b4a82ac2d43a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.401375 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.401281    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.401375 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.401308    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-web-config\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.401375 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.401337    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.401375 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.401365    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.401541 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.401393    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.401541 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.401422    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trj8t\" (UniqueName: \"kubernetes.io/projected/350f2760-583f-4109-9d15-b4a82ac2d43a-kube-api-access-trj8t\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.401541 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.401506    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.442024 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.441965    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 22:10:26.442024 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.441970    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 22:10:26.442024 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.441987    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 22:10:26.449890 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.449870    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 22:10:26.449890 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.449881    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 22:10:26.450053 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.449904    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 22:10:26.450171 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.450154    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qknqm\""
Oct 30 22:10:26.450231 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.450179    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 22:10:26.450231 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.450188    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 22:10:26.450231 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.450180    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 22:10:26.450321 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.450255    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 22:10:26.450321 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.450264    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 22:10:26.450321 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.450285    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 22:10:26.450321 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.450290    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 22:10:26.450432 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.450325    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bq6t45e7eerjf\""
Oct 30 22:10:26.493204 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.493180    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:10:26.502096 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502071    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/350f2760-583f-4109-9d15-b4a82ac2d43a-config-out\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.502198 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502098    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.502198 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502119    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-config\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.502198 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502133    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.502198 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502172    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.502394 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502215    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.502394 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502242    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502274    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502523    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502552    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/350f2760-583f-4109-9d15-b4a82ac2d43a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502578    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502619    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502679    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/350f2760-583f-4109-9d15-b4a82ac2d43a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502722    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502748    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-web-config\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502781    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502808    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502837    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502868    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trj8t\" (UniqueName: \"kubernetes.io/projected/350f2760-583f-4109-9d15-b4a82ac2d43a-kube-api-access-trj8t\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.502910    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.503144    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.503160    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.504718    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.505384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.504792    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.506268 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.505695    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.506268 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.505770    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/350f2760-583f-4109-9d15-b4a82ac2d43a-config-out\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.506268 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.505847    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.506268 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.506099    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-config\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.506570 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.506546    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.506740 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.506588    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.506740 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.506677    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/350f2760-583f-4109-9d15-b4a82ac2d43a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.507546 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.507504    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.508018 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.507973    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.509060 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.509015    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.509800 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.509744    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.510732 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.510700    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-web-config\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.511002 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.510985    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/350f2760-583f-4109-9d15-b4a82ac2d43a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.511741 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.511725    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/350f2760-583f-4109-9d15-b4a82ac2d43a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.542935 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.542913    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 22:10:26.545432 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.545409    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/350f2760-583f-4109-9d15-b4a82ac2d43a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.568405 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.568380    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trj8t\" (UniqueName: \"kubernetes.io/projected/350f2760-583f-4109-9d15-b4a82ac2d43a-kube-api-access-trj8t\") pod \"prometheus-k8s-0\" (UID: \"350f2760-583f-4109-9d15-b4a82ac2d43a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:26.702953 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:26.702883    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:27.135234 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:27.135185    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:10:27.137734 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:10:27.137707    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod350f2760_583f_4109_9d15_b4a82ac2d43a.slice/crio-4aa1f3e34fc8855dc79b663e24ad68573ff131516eaef5ffa6b9fb657ea8481d WatchSource:0}: Error finding container 4aa1f3e34fc8855dc79b663e24ad68573ff131516eaef5ffa6b9fb657ea8481d: Status 404 returned error can't find the container with id 4aa1f3e34fc8855dc79b663e24ad68573ff131516eaef5ffa6b9fb657ea8481d
Oct 30 22:10:27.170845 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:27.170814    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"350f2760-583f-4109-9d15-b4a82ac2d43a","Type":"ContainerStarted","Data":"4aa1f3e34fc8855dc79b663e24ad68573ff131516eaef5ffa6b9fb657ea8481d"}
Oct 30 22:10:27.735811 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:27.735779    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b127d4c4-4878-4467-b936-060ee2bcca83" path="/var/lib/kubelet/pods/b127d4c4-4878-4467-b936-060ee2bcca83/volumes"
Oct 30 22:10:28.177974 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:28.177886    5896 generic.go:356] "Generic (PLEG): container finished" podID="350f2760-583f-4109-9d15-b4a82ac2d43a" containerID="5e3cac6015a7b6885285ebfc715cc7d1dd8589e06ddf1ee58c60ce0b3e147563" exitCode=0
Oct 30 22:10:28.178367 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:28.177968    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"350f2760-583f-4109-9d15-b4a82ac2d43a","Type":"ContainerDied","Data":"5e3cac6015a7b6885285ebfc715cc7d1dd8589e06ddf1ee58c60ce0b3e147563"}
Oct 30 22:10:29.185546 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:29.185506    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"350f2760-583f-4109-9d15-b4a82ac2d43a","Type":"ContainerStarted","Data":"6c622576a352fe8e4abfdecb2e2349948b8520e0da22a7a6b4d621b22f91b9fe"}
Oct 30 22:10:29.185958 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:29.185589    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"350f2760-583f-4109-9d15-b4a82ac2d43a","Type":"ContainerStarted","Data":"a8a31673605ff6dd0c99c93954ca80000e44715914afe3967317cc7c43766549"}
Oct 30 22:10:29.185958 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:29.185632    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"350f2760-583f-4109-9d15-b4a82ac2d43a","Type":"ContainerStarted","Data":"c858d677f11ab32d2c4c0dd8203346a905d2b2f6605955c34a7e1d132be76300"}
Oct 30 22:10:29.185958 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:29.185696    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"350f2760-583f-4109-9d15-b4a82ac2d43a","Type":"ContainerStarted","Data":"1bf94620237156eed70dd19ee95993e339e3825089499284ece6b3a6bd62c38c"}
Oct 30 22:10:29.185958 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:29.185734    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"350f2760-583f-4109-9d15-b4a82ac2d43a","Type":"ContainerStarted","Data":"0121e8f8a3c47bcf0b25b73a8017ffdf4ba78ab4d004d3cfff308452c3a73cb3"}
Oct 30 22:10:29.185958 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:29.185773    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"350f2760-583f-4109-9d15-b4a82ac2d43a","Type":"ContainerStarted","Data":"d3c7d48d0cbf61171d129cca7407424933572f56e0cd90e4fee63060de55b85f"}
Oct 30 22:10:29.333778 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:29.333721    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.319116975 podStartE2EDuration="3.333706775s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.5898ms" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:29.331794834 +0000 UTC m=+81.743260805" watchObservedRunningTime="2026-10-30 22:10:29.333706775 +0000 UTC m=+81.745172742"
Oct 30 22:10:31.703294 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:10:31.703254    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:10:32.628108 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:10:32.628073    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:11:02.626886 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:11:02.626799    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:11:26.703730 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:11:26.703691    5896 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:11:26.720874 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:11:26.720845    5896 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:11:27.429961 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:11:27.429928    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:11:32.628130 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:11:32.628088    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:12:02.626439 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:12:02.626404    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:12:32.629577 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:12:32.629541    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:13:02.626445 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:13:02.626407    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:13:32.629056 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:13:32.629016    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:14:02.626376 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:14:02.626289    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:14:07.779351 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:14:07.779317    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:14:07.784240 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:14:07.784221    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:14:32.630283 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:14:32.630248    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:15:02.627118 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:15:02.627078    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:15:32.628138 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:15:32.628054    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:16:02.626549 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:16:02.626511    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:16:13.346887 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:13.346852    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/s3-init-qnmhz"]
Oct 30 22:16:13.350335 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:13.350315    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-qnmhz"
Oct 30 22:16:13.404852 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:13.404830    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Oct 30 22:16:13.420187 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:13.420165    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxlzj\" (UniqueName: \"kubernetes.io/projected/29d270ce-d373-40ad-b591-8769e420becd-kube-api-access-dxlzj\") pod \"s3-init-qnmhz\" (UID: \"29d270ce-d373-40ad-b591-8769e420becd\") " pod="kserve/s3-init-qnmhz"
Oct 30 22:16:13.420296 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:13.420208    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29d270ce-d373-40ad-b591-8769e420becd-model-cache\") pod \"s3-init-qnmhz\" (UID: \"29d270ce-d373-40ad-b591-8769e420becd\") " pod="kserve/s3-init-qnmhz"
Oct 30 22:16:13.454575 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:13.454548    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Oct 30 22:16:13.454575 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:13.454560    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\""
Oct 30 22:16:13.454769 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:13.454602    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Oct 30 22:16:13.454769 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:13.454668    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-vf6ms\""
Oct 30 22:16:13.455603 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:13.455582    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-qnmhz"]
Oct 30 22:16:13.521515 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:13.521493    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxlzj\" (UniqueName: \"kubernetes.io/projected/29d270ce-d373-40ad-b591-8769e420becd-kube-api-access-dxlzj\") pod \"s3-init-qnmhz\" (UID: \"29d270ce-d373-40ad-b591-8769e420becd\") " pod="kserve/s3-init-qnmhz"
Oct 30 22:16:13.521600 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:13.521535    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29d270ce-d373-40ad-b591-8769e420becd-model-cache\") pod \"s3-init-qnmhz\" (UID: \"29d270ce-d373-40ad-b591-8769e420becd\") " pod="kserve/s3-init-qnmhz"
Oct 30 22:16:13.521856 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:13.521842    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29d270ce-d373-40ad-b591-8769e420becd-model-cache\") pod \"s3-init-qnmhz\" (UID: \"29d270ce-d373-40ad-b591-8769e420becd\") " pod="kserve/s3-init-qnmhz"
Oct 30 22:16:13.585174 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:13.585156    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxlzj\" (UniqueName: \"kubernetes.io/projected/29d270ce-d373-40ad-b591-8769e420becd-kube-api-access-dxlzj\") pod \"s3-init-qnmhz\" (UID: \"29d270ce-d373-40ad-b591-8769e420becd\") " pod="kserve/s3-init-qnmhz"
Oct 30 22:16:13.679185 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:13.679125    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-qnmhz"
Oct 30 22:16:14.049191 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:14.049163    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-qnmhz"]
Oct 30 22:16:14.052343 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:16:14.052316    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29d270ce_d373_40ad_b591_8769e420becd.slice/crio-8903534eb41f56d9d337645cf13f4fc9930bea8c073ebf1db1fc5d1ecf5966cb WatchSource:0}: Error finding container 8903534eb41f56d9d337645cf13f4fc9930bea8c073ebf1db1fc5d1ecf5966cb: Status 404 returned error can't find the container with id 8903534eb41f56d9d337645cf13f4fc9930bea8c073ebf1db1fc5d1ecf5966cb
Oct 30 22:16:14.054231 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:14.054211    5896 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:16:14.485708 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:14.485673    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-qnmhz" event={"ID":"29d270ce-d373-40ad-b591-8769e420becd","Type":"ContainerStarted","Data":"8903534eb41f56d9d337645cf13f4fc9930bea8c073ebf1db1fc5d1ecf5966cb"}
Oct 30 22:16:25.536133 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:25.536093    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-qnmhz" event={"ID":"29d270ce-d373-40ad-b591-8769e420becd","Type":"ContainerStarted","Data":"b5760fe95a30693aa864045e8a4d0e8f22e6ad7739105348ffd69eae675e7599"}
Oct 30 22:16:32.632185 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:16:32.632144    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:16:36.579276 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:36.579244    5896 generic.go:356] "Generic (PLEG): container finished" podID="29d270ce-d373-40ad-b591-8769e420becd" containerID="b5760fe95a30693aa864045e8a4d0e8f22e6ad7739105348ffd69eae675e7599" exitCode=1
Oct 30 22:16:36.579691 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:36.579316    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-qnmhz" event={"ID":"29d270ce-d373-40ad-b591-8769e420becd","Type":"ContainerDied","Data":"b5760fe95a30693aa864045e8a4d0e8f22e6ad7739105348ffd69eae675e7599"}
Oct 30 22:16:37.704104 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:37.704080    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-qnmhz"
Oct 30 22:16:37.821908 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:37.821872    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29d270ce-d373-40ad-b591-8769e420becd-kube-api-access-dxlzj\" (UniqueName: \"kubernetes.io/projected/29d270ce-d373-40ad-b591-8769e420becd-kube-api-access-dxlzj\") pod \"29d270ce-d373-40ad-b591-8769e420becd\" (UID: \"29d270ce-d373-40ad-b591-8769e420becd\") "
Oct 30 22:16:37.822059 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:37.821995    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29d270ce-d373-40ad-b591-8769e420becd-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29d270ce-d373-40ad-b591-8769e420becd-model-cache\") pod \"29d270ce-d373-40ad-b591-8769e420becd\" (UID: \"29d270ce-d373-40ad-b591-8769e420becd\") "
Oct 30 22:16:37.824385 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:37.824344    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29d270ce-d373-40ad-b591-8769e420becd-kube-api-access-dxlzj" pod "29d270ce-d373-40ad-b591-8769e420becd" (UID: "29d270ce-d373-40ad-b591-8769e420becd"). InnerVolumeSpecName "kube-api-access-dxlzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:37.842650 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:37.842588    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29d270ce-d373-40ad-b591-8769e420becd-model-cache" pod "29d270ce-d373-40ad-b591-8769e420becd" (UID: "29d270ce-d373-40ad-b591-8769e420becd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:37.922916 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:37.922894    5896 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29d270ce-d373-40ad-b591-8769e420becd-model-cache\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:37.922916 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:37.922917    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxlzj\" (UniqueName: \"kubernetes.io/projected/29d270ce-d373-40ad-b591-8769e420becd-kube-api-access-dxlzj\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:38.588032 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:38.587993    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-qnmhz" event={"ID":"29d270ce-d373-40ad-b591-8769e420becd","Type":"ContainerDied","Data":"8903534eb41f56d9d337645cf13f4fc9930bea8c073ebf1db1fc5d1ecf5966cb"}
Oct 30 22:16:38.588167 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:38.588042    5896 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8903534eb41f56d9d337645cf13f4fc9930bea8c073ebf1db1fc5d1ecf5966cb"
Oct 30 22:16:38.588167 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:38.588008    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-qnmhz"
Oct 30 22:16:38.855697 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:38.855616    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/s3-init-xzdxb"]
Oct 30 22:16:38.856060 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:38.856007    5896 cpu_manager.go:414] "containerMap: removing container" podUID="29d270ce-d373-40ad-b591-8769e420becd" containerName="download-hf-model"
Oct 30 22:16:38.856060 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:38.856019    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29d270ce-d373-40ad-b591-8769e420becd" containerName="download-hf-model"
Oct 30 22:16:38.856130 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:38.856078    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="29d270ce-d373-40ad-b591-8769e420becd" containerName="download-hf-model"
Oct 30 22:16:38.858250 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:38.858234    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-xzdxb"
Oct 30 22:16:38.913745 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:38.913718    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Oct 30 22:16:38.913965 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:38.913947    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Oct 30 22:16:38.914096 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:38.913955    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\""
Oct 30 22:16:38.914096 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:38.914010    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Oct 30 22:16:38.914096 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:38.914054    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-vf6ms\""
Oct 30 22:16:38.919899 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:38.919876    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-xzdxb"]
Oct 30 22:16:39.031127 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:39.031093    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2a11d80-7da4-490c-990a-0cb41e80448f-model-cache\") pod \"s3-init-xzdxb\" (UID: \"f2a11d80-7da4-490c-990a-0cb41e80448f\") " pod="kserve/s3-init-xzdxb"
Oct 30 22:16:39.031282 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:39.031151    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjkf5\" (UniqueName: \"kubernetes.io/projected/f2a11d80-7da4-490c-990a-0cb41e80448f-kube-api-access-qjkf5\") pod \"s3-init-xzdxb\" (UID: \"f2a11d80-7da4-490c-990a-0cb41e80448f\") " pod="kserve/s3-init-xzdxb"
Oct 30 22:16:39.131953 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:39.131879    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjkf5\" (UniqueName: \"kubernetes.io/projected/f2a11d80-7da4-490c-990a-0cb41e80448f-kube-api-access-qjkf5\") pod \"s3-init-xzdxb\" (UID: \"f2a11d80-7da4-490c-990a-0cb41e80448f\") " pod="kserve/s3-init-xzdxb"
Oct 30 22:16:39.131953 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:39.131948    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2a11d80-7da4-490c-990a-0cb41e80448f-model-cache\") pod \"s3-init-xzdxb\" (UID: \"f2a11d80-7da4-490c-990a-0cb41e80448f\") " pod="kserve/s3-init-xzdxb"
Oct 30 22:16:39.132284 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:39.132269    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2a11d80-7da4-490c-990a-0cb41e80448f-model-cache\") pod \"s3-init-xzdxb\" (UID: \"f2a11d80-7da4-490c-990a-0cb41e80448f\") " pod="kserve/s3-init-xzdxb"
Oct 30 22:16:39.196219 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:39.196194    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjkf5\" (UniqueName: \"kubernetes.io/projected/f2a11d80-7da4-490c-990a-0cb41e80448f-kube-api-access-qjkf5\") pod \"s3-init-xzdxb\" (UID: \"f2a11d80-7da4-490c-990a-0cb41e80448f\") " pod="kserve/s3-init-xzdxb"
Oct 30 22:16:39.484216 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:39.484183    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-xzdxb"
Oct 30 22:16:39.866693 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:39.866671    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-xzdxb"]
Oct 30 22:16:39.869160 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:16:39.869136    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2a11d80_7da4_490c_990a_0cb41e80448f.slice/crio-eb3041a8afe4fdbf3374377f8e3f2334dd354f64e2a0f2d9d66ea71d02b2c9c7 WatchSource:0}: Error finding container eb3041a8afe4fdbf3374377f8e3f2334dd354f64e2a0f2d9d66ea71d02b2c9c7: Status 404 returned error can't find the container with id eb3041a8afe4fdbf3374377f8e3f2334dd354f64e2a0f2d9d66ea71d02b2c9c7
Oct 30 22:16:40.597171 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:40.597135    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-xzdxb" event={"ID":"f2a11d80-7da4-490c-990a-0cb41e80448f","Type":"ContainerStarted","Data":"1248fc74664873c92e10b88943abd9435e9ecd7c39a3d58d793faca9b6cfda2e"}
Oct 30 22:16:40.597334 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:40.597189    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-xzdxb" event={"ID":"f2a11d80-7da4-490c-990a-0cb41e80448f","Type":"ContainerStarted","Data":"eb3041a8afe4fdbf3374377f8e3f2334dd354f64e2a0f2d9d66ea71d02b2c9c7"}
Oct 30 22:16:42.607164 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:42.607132    5896 generic.go:356] "Generic (PLEG): container finished" podID="f2a11d80-7da4-490c-990a-0cb41e80448f" containerID="1248fc74664873c92e10b88943abd9435e9ecd7c39a3d58d793faca9b6cfda2e" exitCode=0
Oct 30 22:16:42.607532 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:42.607176    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-xzdxb" event={"ID":"f2a11d80-7da4-490c-990a-0cb41e80448f","Type":"ContainerDied","Data":"1248fc74664873c92e10b88943abd9435e9ecd7c39a3d58d793faca9b6cfda2e"}
Oct 30 22:16:47.633014 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:47.632982    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-xzdxb" event={"ID":"f2a11d80-7da4-490c-990a-0cb41e80448f","Type":"ContainerStarted","Data":"bbd767db641e8ca6e1c125d653daaccd76275db20819c37f74dd03669c7c1b45"}
Oct 30 22:16:47.757897 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:16:47.757846    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/s3-init-xzdxb" podStartSLOduration=2.479119561 podStartE2EDuration="9.757831974s" totalImagesPullingTime="4.848211764s" totalInitContainerRuntime="2.430500649s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:16:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:16:47.75738464 +0000 UTC m=+460.168850621" watchObservedRunningTime="2026-10-30 22:16:47.757831974 +0000 UTC m=+460.169297941"
Oct 30 22:17:02.625768 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:17:02.625728    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:17:32.628013 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:17:32.627971    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:17:52.877848 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:17:52.877819    5896 generic.go:356] "Generic (PLEG): container finished" podID="f2a11d80-7da4-490c-990a-0cb41e80448f" containerID="bbd767db641e8ca6e1c125d653daaccd76275db20819c37f74dd03669c7c1b45" exitCode=0
Oct 30 22:17:52.878208 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:17:52.877903    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-xzdxb" event={"ID":"f2a11d80-7da4-490c-990a-0cb41e80448f","Type":"ContainerDied","Data":"bbd767db641e8ca6e1c125d653daaccd76275db20819c37f74dd03669c7c1b45"}
Oct 30 22:17:54.001264 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:17:54.001240    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-xzdxb"
Oct 30 22:17:54.115452 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:17:54.115420    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2a11d80-7da4-490c-990a-0cb41e80448f-kube-api-access-qjkf5\" (UniqueName: \"kubernetes.io/projected/f2a11d80-7da4-490c-990a-0cb41e80448f-kube-api-access-qjkf5\") pod \"f2a11d80-7da4-490c-990a-0cb41e80448f\" (UID: \"f2a11d80-7da4-490c-990a-0cb41e80448f\") "
Oct 30 22:17:54.115610 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:17:54.115494    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2a11d80-7da4-490c-990a-0cb41e80448f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2a11d80-7da4-490c-990a-0cb41e80448f-model-cache\") pod \"f2a11d80-7da4-490c-990a-0cb41e80448f\" (UID: \"f2a11d80-7da4-490c-990a-0cb41e80448f\") "
Oct 30 22:17:54.117666 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:17:54.117624    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2a11d80-7da4-490c-990a-0cb41e80448f-kube-api-access-qjkf5" pod "f2a11d80-7da4-490c-990a-0cb41e80448f" (UID: "f2a11d80-7da4-490c-990a-0cb41e80448f"). InnerVolumeSpecName "kube-api-access-qjkf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:54.138039 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:17:54.137976    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2a11d80-7da4-490c-990a-0cb41e80448f-model-cache" pod "f2a11d80-7da4-490c-990a-0cb41e80448f" (UID: "f2a11d80-7da4-490c-990a-0cb41e80448f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:54.216834 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:17:54.216808    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjkf5\" (UniqueName: \"kubernetes.io/projected/f2a11d80-7da4-490c-990a-0cb41e80448f-kube-api-access-qjkf5\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:54.216834 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:17:54.216829    5896 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2a11d80-7da4-490c-990a-0cb41e80448f-model-cache\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:54.887337 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:17:54.887309    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-xzdxb"
Oct 30 22:17:54.887337 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:17:54.887315    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-xzdxb" event={"ID":"f2a11d80-7da4-490c-990a-0cb41e80448f","Type":"ContainerDied","Data":"eb3041a8afe4fdbf3374377f8e3f2334dd354f64e2a0f2d9d66ea71d02b2c9c7"}
Oct 30 22:17:54.887337 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:17:54.887347    5896 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb3041a8afe4fdbf3374377f8e3f2334dd354f64e2a0f2d9d66ea71d02b2c9c7"
Oct 30 22:18:02.625920 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:18:02.625885    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:18:04.300227 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.300197    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/s3-tls-init-custom-tn8wc"]
Oct 30 22:18:04.302560 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.300517    5896 cpu_manager.go:414] "containerMap: removing container" podUID="f2a11d80-7da4-490c-990a-0cb41e80448f" containerName="download-hf-model"
Oct 30 22:18:04.302560 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.300528    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2a11d80-7da4-490c-990a-0cb41e80448f" containerName="download-hf-model"
Oct 30 22:18:04.302560 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.300556    5896 cpu_manager.go:414] "containerMap: removing container" podUID="f2a11d80-7da4-490c-990a-0cb41e80448f" containerName="s3-init"
Oct 30 22:18:04.302560 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.300561    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2a11d80-7da4-490c-990a-0cb41e80448f" containerName="s3-init"
Oct 30 22:18:04.302560 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.300619    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2a11d80-7da4-490c-990a-0cb41e80448f" containerName="s3-init"
Oct 30 22:18:04.303425 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.303409    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-tn8wc"
Oct 30 22:18:04.358980 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.358953    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\""
Oct 30 22:18:04.396243 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.396216    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47f4b8cd-32cc-481b-bdb8-f179eb993882-model-cache\") pod \"s3-tls-init-custom-tn8wc\" (UID: \"47f4b8cd-32cc-481b-bdb8-f179eb993882\") " pod="kserve/s3-tls-init-custom-tn8wc"
Oct 30 22:18:04.396356 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.396316    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc2gj\" (UniqueName: \"kubernetes.io/projected/47f4b8cd-32cc-481b-bdb8-f179eb993882-kube-api-access-kc2gj\") pod \"s3-tls-init-custom-tn8wc\" (UID: \"47f4b8cd-32cc-481b-bdb8-f179eb993882\") " pod="kserve/s3-tls-init-custom-tn8wc"
Oct 30 22:18:04.409680 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.409635    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-vf6ms\""
Oct 30 22:18:04.409809 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.409691    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Oct 30 22:18:04.409809 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.409718    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Oct 30 22:18:04.409809 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.409792    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Oct 30 22:18:04.410513 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.410497    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-custom-tn8wc"]
Oct 30 22:18:04.497079 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.497053    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47f4b8cd-32cc-481b-bdb8-f179eb993882-model-cache\") pod \"s3-tls-init-custom-tn8wc\" (UID: \"47f4b8cd-32cc-481b-bdb8-f179eb993882\") " pod="kserve/s3-tls-init-custom-tn8wc"
Oct 30 22:18:04.497179 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.497104    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc2gj\" (UniqueName: \"kubernetes.io/projected/47f4b8cd-32cc-481b-bdb8-f179eb993882-kube-api-access-kc2gj\") pod \"s3-tls-init-custom-tn8wc\" (UID: \"47f4b8cd-32cc-481b-bdb8-f179eb993882\") " pod="kserve/s3-tls-init-custom-tn8wc"
Oct 30 22:18:04.497391 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.497374    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47f4b8cd-32cc-481b-bdb8-f179eb993882-model-cache\") pod \"s3-tls-init-custom-tn8wc\" (UID: \"47f4b8cd-32cc-481b-bdb8-f179eb993882\") " pod="kserve/s3-tls-init-custom-tn8wc"
Oct 30 22:18:04.561214 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.561162    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc2gj\" (UniqueName: \"kubernetes.io/projected/47f4b8cd-32cc-481b-bdb8-f179eb993882-kube-api-access-kc2gj\") pod \"s3-tls-init-custom-tn8wc\" (UID: \"47f4b8cd-32cc-481b-bdb8-f179eb993882\") " pod="kserve/s3-tls-init-custom-tn8wc"
Oct 30 22:18:04.625875 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.625854    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-tn8wc"
Oct 30 22:18:04.997070 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:04.997047    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-custom-tn8wc"]
Oct 30 22:18:04.999743 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:18:04.999716    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47f4b8cd_32cc_481b_bdb8_f179eb993882.slice/crio-540770480e6559d728f1a7eeaee941cd4617ab68d8074cc2f12a2c79c14f5cec WatchSource:0}: Error finding container 540770480e6559d728f1a7eeaee941cd4617ab68d8074cc2f12a2c79c14f5cec: Status 404 returned error can't find the container with id 540770480e6559d728f1a7eeaee941cd4617ab68d8074cc2f12a2c79c14f5cec
Oct 30 22:18:05.929976 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:05.929940    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-tn8wc" event={"ID":"47f4b8cd-32cc-481b-bdb8-f179eb993882","Type":"ContainerStarted","Data":"57d80e0bc967cd4d49484f9f4d07612ad01ca4f1b5cdafb0d82d676269bbde36"}
Oct 30 22:18:05.930374 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:05.929988    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-tn8wc" event={"ID":"47f4b8cd-32cc-481b-bdb8-f179eb993882","Type":"ContainerStarted","Data":"540770480e6559d728f1a7eeaee941cd4617ab68d8074cc2f12a2c79c14f5cec"}
Oct 30 22:18:07.940654 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:07.940608    5896 generic.go:356] "Generic (PLEG): container finished" podID="47f4b8cd-32cc-481b-bdb8-f179eb993882" containerID="57d80e0bc967cd4d49484f9f4d07612ad01ca4f1b5cdafb0d82d676269bbde36" exitCode=0
Oct 30 22:18:07.940993 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:07.940680    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-tn8wc" event={"ID":"47f4b8cd-32cc-481b-bdb8-f179eb993882","Type":"ContainerDied","Data":"57d80e0bc967cd4d49484f9f4d07612ad01ca4f1b5cdafb0d82d676269bbde36"}
Oct 30 22:18:08.045523 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:08.045499    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Oct 30 22:18:08.945898 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:08.945865    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-tn8wc" event={"ID":"47f4b8cd-32cc-481b-bdb8-f179eb993882","Type":"ContainerStarted","Data":"be7d8af0ddd5bd59ef2d810cd510a98233bf27439404de99b217a770df3c6d9a"}
Oct 30 22:18:09.088359 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:09.088310    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/s3-tls-init-custom-tn8wc" podStartSLOduration=2.936216726 podStartE2EDuration="5.088296161s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.152079435s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:09.087689558 +0000 UTC m=+541.499155524" watchObservedRunningTime="2026-10-30 22:18:09.088296161 +0000 UTC m=+541.499762186"
Oct 30 22:18:14.973222 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:14.973192    5896 generic.go:356] "Generic (PLEG): container finished" podID="47f4b8cd-32cc-481b-bdb8-f179eb993882" containerID="be7d8af0ddd5bd59ef2d810cd510a98233bf27439404de99b217a770df3c6d9a" exitCode=0
Oct 30 22:18:14.973578 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:14.973254    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-tn8wc" event={"ID":"47f4b8cd-32cc-481b-bdb8-f179eb993882","Type":"ContainerDied","Data":"be7d8af0ddd5bd59ef2d810cd510a98233bf27439404de99b217a770df3c6d9a"}
Oct 30 22:18:16.122392 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:16.122370    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-tn8wc"
Oct 30 22:18:16.194891 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:16.194866    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47f4b8cd-32cc-481b-bdb8-f179eb993882-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47f4b8cd-32cc-481b-bdb8-f179eb993882-model-cache\") pod \"47f4b8cd-32cc-481b-bdb8-f179eb993882\" (UID: \"47f4b8cd-32cc-481b-bdb8-f179eb993882\") "
Oct 30 22:18:16.194989 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:16.194910    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47f4b8cd-32cc-481b-bdb8-f179eb993882-kube-api-access-kc2gj\" (UniqueName: \"kubernetes.io/projected/47f4b8cd-32cc-481b-bdb8-f179eb993882-kube-api-access-kc2gj\") pod \"47f4b8cd-32cc-481b-bdb8-f179eb993882\" (UID: \"47f4b8cd-32cc-481b-bdb8-f179eb993882\") "
Oct 30 22:18:16.196998 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:16.196967    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47f4b8cd-32cc-481b-bdb8-f179eb993882-kube-api-access-kc2gj" pod "47f4b8cd-32cc-481b-bdb8-f179eb993882" (UID: "47f4b8cd-32cc-481b-bdb8-f179eb993882"). InnerVolumeSpecName "kube-api-access-kc2gj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:16.217984 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:16.217958    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47f4b8cd-32cc-481b-bdb8-f179eb993882-model-cache" pod "47f4b8cd-32cc-481b-bdb8-f179eb993882" (UID: "47f4b8cd-32cc-481b-bdb8-f179eb993882"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:16.295720 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:16.295666    5896 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47f4b8cd-32cc-481b-bdb8-f179eb993882-model-cache\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:16.295720 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:16.295688    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kc2gj\" (UniqueName: \"kubernetes.io/projected/47f4b8cd-32cc-481b-bdb8-f179eb993882-kube-api-access-kc2gj\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:16.983065 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:16.983026    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-tn8wc" event={"ID":"47f4b8cd-32cc-481b-bdb8-f179eb993882","Type":"ContainerDied","Data":"540770480e6559d728f1a7eeaee941cd4617ab68d8074cc2f12a2c79c14f5cec"}
Oct 30 22:18:16.983065 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:16.983069    5896 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="540770480e6559d728f1a7eeaee941cd4617ab68d8074cc2f12a2c79c14f5cec"
Oct 30 22:18:16.983258 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:16.983045    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-tn8wc"
Oct 30 22:18:23.487382 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.487350    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/s3-tls-init-serving-f78dx"]
Oct 30 22:18:23.487792 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.487691    5896 cpu_manager.go:414] "containerMap: removing container" podUID="47f4b8cd-32cc-481b-bdb8-f179eb993882" containerName="download-hf-model"
Oct 30 22:18:23.487792 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.487703    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47f4b8cd-32cc-481b-bdb8-f179eb993882" containerName="download-hf-model"
Oct 30 22:18:23.487792 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.487726    5896 cpu_manager.go:414] "containerMap: removing container" podUID="47f4b8cd-32cc-481b-bdb8-f179eb993882" containerName="s3-tls-init-custom"
Oct 30 22:18:23.487792 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.487731    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47f4b8cd-32cc-481b-bdb8-f179eb993882" containerName="s3-tls-init-custom"
Oct 30 22:18:23.487792 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.487791    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="47f4b8cd-32cc-481b-bdb8-f179eb993882" containerName="s3-tls-init-custom"
Oct 30 22:18:23.490730 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.490715    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-f78dx"
Oct 30 22:18:23.547680 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.547633    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Oct 30 22:18:23.548908 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.548889    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/456d20bf-584c-4b3e-b801-6a14791b0a48-model-cache\") pod \"s3-tls-init-serving-f78dx\" (UID: \"456d20bf-584c-4b3e-b801-6a14791b0a48\") " pod="kserve/s3-tls-init-serving-f78dx"
Oct 30 22:18:23.548978 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.548919    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5mtf\" (UniqueName: \"kubernetes.io/projected/456d20bf-584c-4b3e-b801-6a14791b0a48-kube-api-access-j5mtf\") pod \"s3-tls-init-serving-f78dx\" (UID: \"456d20bf-584c-4b3e-b801-6a14791b0a48\") " pod="kserve/s3-tls-init-serving-f78dx"
Oct 30 22:18:23.597151 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.597128    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Oct 30 22:18:23.597295 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.597276    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\""
Oct 30 22:18:23.597348 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.597286    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-serving-artifact\""
Oct 30 22:18:23.597348 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.597340    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-vf6ms\""
Oct 30 22:18:23.598183 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.598168    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-serving-f78dx"]
Oct 30 22:18:23.650304 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.650281    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/456d20bf-584c-4b3e-b801-6a14791b0a48-model-cache\") pod \"s3-tls-init-serving-f78dx\" (UID: \"456d20bf-584c-4b3e-b801-6a14791b0a48\") " pod="kserve/s3-tls-init-serving-f78dx"
Oct 30 22:18:23.650443 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.650311    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5mtf\" (UniqueName: \"kubernetes.io/projected/456d20bf-584c-4b3e-b801-6a14791b0a48-kube-api-access-j5mtf\") pod \"s3-tls-init-serving-f78dx\" (UID: \"456d20bf-584c-4b3e-b801-6a14791b0a48\") " pod="kserve/s3-tls-init-serving-f78dx"
Oct 30 22:18:23.650704 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.650684    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/456d20bf-584c-4b3e-b801-6a14791b0a48-model-cache\") pod \"s3-tls-init-serving-f78dx\" (UID: \"456d20bf-584c-4b3e-b801-6a14791b0a48\") " pod="kserve/s3-tls-init-serving-f78dx"
Oct 30 22:18:23.714710 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.714684    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5mtf\" (UniqueName: \"kubernetes.io/projected/456d20bf-584c-4b3e-b801-6a14791b0a48-kube-api-access-j5mtf\") pod \"s3-tls-init-serving-f78dx\" (UID: \"456d20bf-584c-4b3e-b801-6a14791b0a48\") " pod="kserve/s3-tls-init-serving-f78dx"
Oct 30 22:18:23.816870 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:23.816812    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-f78dx"
Oct 30 22:18:24.189295 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:24.189217    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-serving-f78dx"]
Oct 30 22:18:24.192353 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:18:24.192325    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod456d20bf_584c_4b3e_b801_6a14791b0a48.slice/crio-fd67197d3a082c6677f2859b56e870dc0617174c4c90a162d03fd966e4eeb3a5 WatchSource:0}: Error finding container fd67197d3a082c6677f2859b56e870dc0617174c4c90a162d03fd966e4eeb3a5: Status 404 returned error can't find the container with id fd67197d3a082c6677f2859b56e870dc0617174c4c90a162d03fd966e4eeb3a5
Oct 30 22:18:25.014902 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:25.014859    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-f78dx" event={"ID":"456d20bf-584c-4b3e-b801-6a14791b0a48","Type":"ContainerStarted","Data":"427f8837421e454045d011b13a9a1d733f99418b1d96f947cabe37ae1df8ceda"}
Oct 30 22:18:25.015283 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:25.014907    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-f78dx" event={"ID":"456d20bf-584c-4b3e-b801-6a14791b0a48","Type":"ContainerStarted","Data":"fd67197d3a082c6677f2859b56e870dc0617174c4c90a162d03fd966e4eeb3a5"}
Oct 30 22:18:27.025554 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:27.025523    5896 generic.go:356] "Generic (PLEG): container finished" podID="456d20bf-584c-4b3e-b801-6a14791b0a48" containerID="427f8837421e454045d011b13a9a1d733f99418b1d96f947cabe37ae1df8ceda" exitCode=0
Oct 30 22:18:27.025932 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:27.025597    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-f78dx" event={"ID":"456d20bf-584c-4b3e-b801-6a14791b0a48","Type":"ContainerDied","Data":"427f8837421e454045d011b13a9a1d733f99418b1d96f947cabe37ae1df8ceda"}
Oct 30 22:18:28.031316 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:28.031279    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-f78dx" event={"ID":"456d20bf-584c-4b3e-b801-6a14791b0a48","Type":"ContainerStarted","Data":"fbc8c923c8876bc67a8f7ea5ef489818ae4ac4f3f84798ba72e592fc2d646b80"}
Oct 30 22:18:28.150703 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:28.150655    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/s3-tls-init-serving-f78dx" podStartSLOduration=2.934959987 podStartE2EDuration="5.150624317s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.21566433s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:28.149784574 +0000 UTC m=+560.561250542" watchObservedRunningTime="2026-10-30 22:18:28.150624317 +0000 UTC m=+560.562090261"
Oct 30 22:18:32.626404 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:18:32.626357    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:18:33.053704 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:33.053675    5896 generic.go:356] "Generic (PLEG): container finished" podID="456d20bf-584c-4b3e-b801-6a14791b0a48" containerID="fbc8c923c8876bc67a8f7ea5ef489818ae4ac4f3f84798ba72e592fc2d646b80" exitCode=0
Oct 30 22:18:33.053854 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:33.053710    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-f78dx" event={"ID":"456d20bf-584c-4b3e-b801-6a14791b0a48","Type":"ContainerDied","Data":"fbc8c923c8876bc67a8f7ea5ef489818ae4ac4f3f84798ba72e592fc2d646b80"}
Oct 30 22:18:34.192570 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:34.192542    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-f78dx"
Oct 30 22:18:34.340221 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:34.340149    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/456d20bf-584c-4b3e-b801-6a14791b0a48-kube-api-access-j5mtf\" (UniqueName: \"kubernetes.io/projected/456d20bf-584c-4b3e-b801-6a14791b0a48-kube-api-access-j5mtf\") pod \"456d20bf-584c-4b3e-b801-6a14791b0a48\" (UID: \"456d20bf-584c-4b3e-b801-6a14791b0a48\") "
Oct 30 22:18:34.340221 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:34.340206    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/456d20bf-584c-4b3e-b801-6a14791b0a48-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/456d20bf-584c-4b3e-b801-6a14791b0a48-model-cache\") pod \"456d20bf-584c-4b3e-b801-6a14791b0a48\" (UID: \"456d20bf-584c-4b3e-b801-6a14791b0a48\") "
Oct 30 22:18:34.342336 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:34.342312    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/456d20bf-584c-4b3e-b801-6a14791b0a48-kube-api-access-j5mtf" pod "456d20bf-584c-4b3e-b801-6a14791b0a48" (UID: "456d20bf-584c-4b3e-b801-6a14791b0a48"). InnerVolumeSpecName "kube-api-access-j5mtf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:34.363912 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:34.363881    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/456d20bf-584c-4b3e-b801-6a14791b0a48-model-cache" pod "456d20bf-584c-4b3e-b801-6a14791b0a48" (UID: "456d20bf-584c-4b3e-b801-6a14791b0a48"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:34.440972 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:34.440940    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5mtf\" (UniqueName: \"kubernetes.io/projected/456d20bf-584c-4b3e-b801-6a14791b0a48-kube-api-access-j5mtf\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:34.440972 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:34.440966    5896 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/456d20bf-584c-4b3e-b801-6a14791b0a48-model-cache\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:35.063844 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:35.063819    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-f78dx"
Oct 30 22:18:35.063994 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:35.063823    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-f78dx" event={"ID":"456d20bf-584c-4b3e-b801-6a14791b0a48","Type":"ContainerDied","Data":"fd67197d3a082c6677f2859b56e870dc0617174c4c90a162d03fd966e4eeb3a5"}
Oct 30 22:18:35.063994 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:35.063928    5896 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd67197d3a082c6677f2859b56e870dc0617174c4c90a162d03fd966e4eeb3a5"
Oct 30 22:18:57.073236 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.073198    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"]
Oct 30 22:18:57.073728 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.073675    5896 cpu_manager.go:414] "containerMap: removing container" podUID="456d20bf-584c-4b3e-b801-6a14791b0a48" containerName="s3-tls-init-serving"
Oct 30 22:18:57.073728 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.073691    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="456d20bf-584c-4b3e-b801-6a14791b0a48" containerName="s3-tls-init-serving"
Oct 30 22:18:57.073728 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.073714    5896 cpu_manager.go:414] "containerMap: removing container" podUID="456d20bf-584c-4b3e-b801-6a14791b0a48" containerName="download-hf-model"
Oct 30 22:18:57.073728 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.073721    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="456d20bf-584c-4b3e-b801-6a14791b0a48" containerName="download-hf-model"
Oct 30 22:18:57.073860 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.073802    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="456d20bf-584c-4b3e-b801-6a14791b0a48" containerName="s3-tls-init-serving"
Oct 30 22:18:57.076920 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.076905    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.181919 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.181890    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"seaweedfs-s3-creds\""
Oct 30 22:18:57.181919 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.181905    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-pmml-v2-kserve-kube-rbac-proxy-sar-config\""
Oct 30 22:18:57.181919 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.181890    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"default-dockercfg-nnb5w\""
Oct 30 22:18:57.182130 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.181922    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"openshift-service-ca.crt\""
Oct 30 22:18:57.182130 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.181904    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"kube-root-ca.crt\""
Oct 30 22:18:57.182130 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.181955    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:18:57.182130 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.182031    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-pmml-v2-kserve-predictor-serving-cert\""
Oct 30 22:18:57.183154 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.183138    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"]
Oct 30 22:18:57.215627 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.215600    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/36175c33-08f8-4af4-86fb-feb017a271fd-proxy-tls\") pod \"isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") " pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.215743 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.215627    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/36175c33-08f8-4af4-86fb-feb017a271fd-cabundle-cert\") pod \"isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") " pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.215791 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.215745    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/36175c33-08f8-4af4-86fb-feb017a271fd-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") " pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.215826 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.215806    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/36175c33-08f8-4af4-86fb-feb017a271fd-kserve-provision-location\") pod \"isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") " pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.215860 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.215843    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kjdl\" (UniqueName: \"kubernetes.io/projected/36175c33-08f8-4af4-86fb-feb017a271fd-kube-api-access-9kjdl\") pod \"isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") " pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.317155 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.317121    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/36175c33-08f8-4af4-86fb-feb017a271fd-proxy-tls\") pod \"isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") " pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.317155 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.317155    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/36175c33-08f8-4af4-86fb-feb017a271fd-cabundle-cert\") pod \"isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") " pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.317374 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.317203    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/36175c33-08f8-4af4-86fb-feb017a271fd-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") " pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.317374 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.317221    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/36175c33-08f8-4af4-86fb-feb017a271fd-kserve-provision-location\") pod \"isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") " pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.317374 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.317247    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kjdl\" (UniqueName: \"kubernetes.io/projected/36175c33-08f8-4af4-86fb-feb017a271fd-kube-api-access-9kjdl\") pod \"isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") " pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.317753 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.317726    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/36175c33-08f8-4af4-86fb-feb017a271fd-kserve-provision-location\") pod \"isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") " pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.317945 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.317925    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/36175c33-08f8-4af4-86fb-feb017a271fd-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") " pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.318013 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.317965    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/36175c33-08f8-4af4-86fb-feb017a271fd-cabundle-cert\") pod \"isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") " pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.319781 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.319763    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/36175c33-08f8-4af4-86fb-feb017a271fd-proxy-tls\") pod \"isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") " pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.380844 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.380794    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kjdl\" (UniqueName: \"kubernetes.io/projected/36175c33-08f8-4af4-86fb-feb017a271fd-kube-api-access-9kjdl\") pod \"isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") " pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.393674 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.393657    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:18:57.816806 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:57.816776    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"]
Oct 30 22:18:57.819788 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:18:57.819759    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36175c33_08f8_4af4_86fb_feb017a271fd.slice/crio-552602c8e0609b5e1af1d63da92f4d3e85660cb2d2fff6156f72bf885608689a WatchSource:0}: Error finding container 552602c8e0609b5e1af1d63da92f4d3e85660cb2d2fff6156f72bf885608689a: Status 404 returned error can't find the container with id 552602c8e0609b5e1af1d63da92f4d3e85660cb2d2fff6156f72bf885608689a
Oct 30 22:18:58.161005 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:58.160922    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" event={"ID":"36175c33-08f8-4af4-86fb-feb017a271fd","Type":"ContainerStarted","Data":"d5398a4a3d33ecc92ee257b8c3285634abd137ea22a77215f83adc8b72298a46"}
Oct 30 22:18:58.161383 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:18:58.161008    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" event={"ID":"36175c33-08f8-4af4-86fb-feb017a271fd","Type":"ContainerStarted","Data":"552602c8e0609b5e1af1d63da92f4d3e85660cb2d2fff6156f72bf885608689a"}
Oct 30 22:19:02.178750 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:19:02.178671    5896 generic.go:356] "Generic (PLEG): container finished" podID="36175c33-08f8-4af4-86fb-feb017a271fd" containerID="d5398a4a3d33ecc92ee257b8c3285634abd137ea22a77215f83adc8b72298a46" exitCode=0
Oct 30 22:19:02.178750 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:19:02.178707    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" event={"ID":"36175c33-08f8-4af4-86fb-feb017a271fd","Type":"ContainerDied","Data":"d5398a4a3d33ecc92ee257b8c3285634abd137ea22a77215f83adc8b72298a46"}
Oct 30 22:19:02.625839 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:19:02.625805    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:19:07.811679 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:19:07.811617    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:19:07.818231 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:19:07.818210    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:19:11.221844 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:19:11.221812    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" event={"ID":"36175c33-08f8-4af4-86fb-feb017a271fd","Type":"ContainerStarted","Data":"4d428e0070894ea5b6d264ded193bdc677189daa718946be7cc7e506906b7fd8"}
Oct 30 22:19:14.238509 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:19:14.238468    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" event={"ID":"36175c33-08f8-4af4-86fb-feb017a271fd","Type":"ContainerStarted","Data":"c5d3d5094227cbb1d1ac0ff7145626ce18273f95961773e6f9c71bf2f0c31f03"}
Oct 30 22:19:14.238920 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:19:14.238732    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:19:14.238920 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:19:14.238865    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:19:14.240107 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:19:14.240075    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:19:14.412693 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:19:14.412622    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" podStartSLOduration=2.052683246 podStartE2EDuration="17.412605953s" totalImagesPullingTime="11.447142598s" totalInitContainerRuntime="3.912780109s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:14.411598111 +0000 UTC m=+606.823064101" watchObservedRunningTime="2026-10-30 22:19:14.412605953 +0000 UTC m=+606.824071922"
Oct 30 22:19:15.242914 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:19:15.242876    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:19:20.247506 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:19:20.247478    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:19:20.248071 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:19:20.248045    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:19:30.247952 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:19:30.247910    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:19:32.628873 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:19:32.628830    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:19:40.248127 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:19:40.248089    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:19:50.248894 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:19:50.248813    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:20:00.248508 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:00.248464    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:20:02.627294 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:20:02.627254    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:20:10.248774 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:10.248733    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:20:20.248589 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:20.248554    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:20:30.248823 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:30.248783    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:20:32.628427 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:20:32.628383    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:20:40.249223 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:40.249193    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:20:48.713604 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:48.713579    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"]
Oct 30 22:20:48.714092 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:48.713808    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kserve-container" containerID="cri-o://4d428e0070894ea5b6d264ded193bdc677189daa718946be7cc7e506906b7fd8" gracePeriod=30
Oct 30 22:20:48.714092 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:48.713862    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kube-rbac-proxy" containerID="cri-o://c5d3d5094227cbb1d1ac0ff7145626ce18273f95961773e6f9c71bf2f0c31f03" gracePeriod=30
Oct 30 22:20:49.624964 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:49.624929    5896 generic.go:356] "Generic (PLEG): container finished" podID="36175c33-08f8-4af4-86fb-feb017a271fd" containerID="c5d3d5094227cbb1d1ac0ff7145626ce18273f95961773e6f9c71bf2f0c31f03" exitCode=0
Oct 30 22:20:49.625133 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:49.624998    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" event={"ID":"36175c33-08f8-4af4-86fb-feb017a271fd","Type":"ContainerDied","Data":"c5d3d5094227cbb1d1ac0ff7145626ce18273f95961773e6f9c71bf2f0c31f03"}
Oct 30 22:20:50.243755 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:50.243719    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.24:8643/healthz\": dial tcp 10.134.0.24:8643: connect: connection refused"
Oct 30 22:20:50.248048 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:50.248026    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:20:52.352009 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.351984    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:20:52.525387 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.525363    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36175c33-08f8-4af4-86fb-feb017a271fd-kube-api-access-9kjdl\" (UniqueName: \"kubernetes.io/projected/36175c33-08f8-4af4-86fb-feb017a271fd-kube-api-access-9kjdl\") pod \"36175c33-08f8-4af4-86fb-feb017a271fd\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") "
Oct 30 22:20:52.525526 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.525398    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36175c33-08f8-4af4-86fb-feb017a271fd-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/36175c33-08f8-4af4-86fb-feb017a271fd-kube-rbac-proxy-sar-config\") pod \"36175c33-08f8-4af4-86fb-feb017a271fd\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") "
Oct 30 22:20:52.525526 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.525419    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36175c33-08f8-4af4-86fb-feb017a271fd-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/36175c33-08f8-4af4-86fb-feb017a271fd-kserve-provision-location\") pod \"36175c33-08f8-4af4-86fb-feb017a271fd\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") "
Oct 30 22:20:52.525617 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.525542    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36175c33-08f8-4af4-86fb-feb017a271fd-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/36175c33-08f8-4af4-86fb-feb017a271fd-cabundle-cert\") pod \"36175c33-08f8-4af4-86fb-feb017a271fd\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") "
Oct 30 22:20:52.525617 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.525576    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36175c33-08f8-4af4-86fb-feb017a271fd-proxy-tls\" (UniqueName: \"kubernetes.io/secret/36175c33-08f8-4af4-86fb-feb017a271fd-proxy-tls\") pod \"36175c33-08f8-4af4-86fb-feb017a271fd\" (UID: \"36175c33-08f8-4af4-86fb-feb017a271fd\") "
Oct 30 22:20:52.525822 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.525785    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36175c33-08f8-4af4-86fb-feb017a271fd-kube-rbac-proxy-sar-config" pod "36175c33-08f8-4af4-86fb-feb017a271fd" (UID: "36175c33-08f8-4af4-86fb-feb017a271fd"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:52.525888 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.525807    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36175c33-08f8-4af4-86fb-feb017a271fd-kserve-provision-location" pod "36175c33-08f8-4af4-86fb-feb017a271fd" (UID: "36175c33-08f8-4af4-86fb-feb017a271fd"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:52.525974 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.525951    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36175c33-08f8-4af4-86fb-feb017a271fd-cabundle-cert" pod "36175c33-08f8-4af4-86fb-feb017a271fd" (UID: "36175c33-08f8-4af4-86fb-feb017a271fd"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:52.527652 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.527620    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36175c33-08f8-4af4-86fb-feb017a271fd-kube-api-access-9kjdl" pod "36175c33-08f8-4af4-86fb-feb017a271fd" (UID: "36175c33-08f8-4af4-86fb-feb017a271fd"). InnerVolumeSpecName "kube-api-access-9kjdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:52.527728 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.527703    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36175c33-08f8-4af4-86fb-feb017a271fd-proxy-tls" pod "36175c33-08f8-4af4-86fb-feb017a271fd" (UID: "36175c33-08f8-4af4-86fb-feb017a271fd"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:52.626308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.626281    5896 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/36175c33-08f8-4af4-86fb-feb017a271fd-cabundle-cert\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:52.626308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.626304    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/36175c33-08f8-4af4-86fb-feb017a271fd-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:52.626462 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.626316    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kjdl\" (UniqueName: \"kubernetes.io/projected/36175c33-08f8-4af4-86fb-feb017a271fd-kube-api-access-9kjdl\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:52.626462 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.626325    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/36175c33-08f8-4af4-86fb-feb017a271fd-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:52.626462 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.626334    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/36175c33-08f8-4af4-86fb-feb017a271fd-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:52.638318 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.638291    5896 generic.go:356] "Generic (PLEG): container finished" podID="36175c33-08f8-4af4-86fb-feb017a271fd" containerID="4d428e0070894ea5b6d264ded193bdc677189daa718946be7cc7e506906b7fd8" exitCode=0
Oct 30 22:20:52.638421 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.638339    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" event={"ID":"36175c33-08f8-4af4-86fb-feb017a271fd","Type":"ContainerDied","Data":"4d428e0070894ea5b6d264ded193bdc677189daa718946be7cc7e506906b7fd8"}
Oct 30 22:20:52.638421 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.638348    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"
Oct 30 22:20:52.638421 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.638385    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg" event={"ID":"36175c33-08f8-4af4-86fb-feb017a271fd","Type":"ContainerDied","Data":"552602c8e0609b5e1af1d63da92f4d3e85660cb2d2fff6156f72bf885608689a"}
Oct 30 22:20:52.638421 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.638411    5896 scope.go:122] "RemoveContainer" containerID="c5d3d5094227cbb1d1ac0ff7145626ce18273f95961773e6f9c71bf2f0c31f03"
Oct 30 22:20:52.646914 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.646890    5896 scope.go:122] "RemoveContainer" containerID="4d428e0070894ea5b6d264ded193bdc677189daa718946be7cc7e506906b7fd8"
Oct 30 22:20:52.654227 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.654212    5896 scope.go:122] "RemoveContainer" containerID="d5398a4a3d33ecc92ee257b8c3285634abd137ea22a77215f83adc8b72298a46"
Oct 30 22:20:52.661692 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.661675    5896 scope.go:122] "RemoveContainer" containerID="c5d3d5094227cbb1d1ac0ff7145626ce18273f95961773e6f9c71bf2f0c31f03"
Oct 30 22:20:52.661974 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:20:52.661955    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5d3d5094227cbb1d1ac0ff7145626ce18273f95961773e6f9c71bf2f0c31f03\": container with ID starting with c5d3d5094227cbb1d1ac0ff7145626ce18273f95961773e6f9c71bf2f0c31f03 not found: ID does not exist" containerID="c5d3d5094227cbb1d1ac0ff7145626ce18273f95961773e6f9c71bf2f0c31f03"
Oct 30 22:20:52.662019 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.661983    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5d3d5094227cbb1d1ac0ff7145626ce18273f95961773e6f9c71bf2f0c31f03"} err="failed to get container status \"c5d3d5094227cbb1d1ac0ff7145626ce18273f95961773e6f9c71bf2f0c31f03\": rpc error: code = NotFound desc = could not find container \"c5d3d5094227cbb1d1ac0ff7145626ce18273f95961773e6f9c71bf2f0c31f03\": container with ID starting with c5d3d5094227cbb1d1ac0ff7145626ce18273f95961773e6f9c71bf2f0c31f03 not found: ID does not exist"
Oct 30 22:20:52.662019 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.662000    5896 scope.go:122] "RemoveContainer" containerID="4d428e0070894ea5b6d264ded193bdc677189daa718946be7cc7e506906b7fd8"
Oct 30 22:20:52.662251 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:20:52.662236    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d428e0070894ea5b6d264ded193bdc677189daa718946be7cc7e506906b7fd8\": container with ID starting with 4d428e0070894ea5b6d264ded193bdc677189daa718946be7cc7e506906b7fd8 not found: ID does not exist" containerID="4d428e0070894ea5b6d264ded193bdc677189daa718946be7cc7e506906b7fd8"
Oct 30 22:20:52.662291 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.662255    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d428e0070894ea5b6d264ded193bdc677189daa718946be7cc7e506906b7fd8"} err="failed to get container status \"4d428e0070894ea5b6d264ded193bdc677189daa718946be7cc7e506906b7fd8\": rpc error: code = NotFound desc = could not find container \"4d428e0070894ea5b6d264ded193bdc677189daa718946be7cc7e506906b7fd8\": container with ID starting with 4d428e0070894ea5b6d264ded193bdc677189daa718946be7cc7e506906b7fd8 not found: ID does not exist"
Oct 30 22:20:52.662291 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.662269    5896 scope.go:122] "RemoveContainer" containerID="d5398a4a3d33ecc92ee257b8c3285634abd137ea22a77215f83adc8b72298a46"
Oct 30 22:20:52.662480 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:20:52.662466    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5398a4a3d33ecc92ee257b8c3285634abd137ea22a77215f83adc8b72298a46\": container with ID starting with d5398a4a3d33ecc92ee257b8c3285634abd137ea22a77215f83adc8b72298a46 not found: ID does not exist" containerID="d5398a4a3d33ecc92ee257b8c3285634abd137ea22a77215f83adc8b72298a46"
Oct 30 22:20:52.662525 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.662485    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5398a4a3d33ecc92ee257b8c3285634abd137ea22a77215f83adc8b72298a46"} err="failed to get container status \"d5398a4a3d33ecc92ee257b8c3285634abd137ea22a77215f83adc8b72298a46\": rpc error: code = NotFound desc = could not find container \"d5398a4a3d33ecc92ee257b8c3285634abd137ea22a77215f83adc8b72298a46\": container with ID starting with d5398a4a3d33ecc92ee257b8c3285634abd137ea22a77215f83adc8b72298a46 not found: ID does not exist"
Oct 30 22:20:52.807869 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.807812    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"]
Oct 30 22:20:52.813501 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:52.813483    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-bc5112d610/isvc-pmml-v2-kserve-predictor-6d87b6dd67-zlxdg"]
Oct 30 22:20:53.734891 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:20:53.734856    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" path="/var/lib/kubelet/pods/36175c33-08f8-4af4-86fb-feb017a271fd/volumes"
Oct 30 22:21:02.628038 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:21:02.627991    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:21:32.630871 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:21:32.630784    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:22:02.627828 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:22:02.627792    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:22:32.628082 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:22:32.628039    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:23:02.626961 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:23:02.626922    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:23:32.628697 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:23:32.628661    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:23:36.277015 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.276986    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"]
Oct 30 22:23:36.277460 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.277305    5896 cpu_manager.go:414] "containerMap: removing container" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="storage-initializer"
Oct 30 22:23:36.277460 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.277315    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="storage-initializer"
Oct 30 22:23:36.277460 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.277324    5896 cpu_manager.go:414] "containerMap: removing container" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kube-rbac-proxy"
Oct 30 22:23:36.277460 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.277329    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kube-rbac-proxy"
Oct 30 22:23:36.277460 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.277340    5896 cpu_manager.go:414] "containerMap: removing container" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kserve-container"
Oct 30 22:23:36.277460 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.277345    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kserve-container"
Oct 30 22:23:36.277460 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.277395    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kube-rbac-proxy"
Oct 30 22:23:36.277460 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.277404    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="36175c33-08f8-4af4-86fb-feb017a271fd" containerName="kserve-container"
Oct 30 22:23:36.279360 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.279341    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:36.319572 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.319541    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/686192b8-37f7-406b-8060-60a99a73d6b5-cabundle-cert\") pod \"isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") " pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:36.319707 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.319592    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/686192b8-37f7-406b-8060-60a99a73d6b5-proxy-tls\") pod \"isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") " pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:36.319770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.319737    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/686192b8-37f7-406b-8060-60a99a73d6b5-kube-rbac-proxy-sar-config\") pod \"isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") " pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:36.319770 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.319763    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb25z\" (UniqueName: \"kubernetes.io/projected/686192b8-37f7-406b-8060-60a99a73d6b5-kube-api-access-zb25z\") pod \"isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") " pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:36.319849 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.319812    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/686192b8-37f7-406b-8060-60a99a73d6b5-kserve-provision-location\") pod \"isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") " pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:36.334228 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.334201    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"openshift-service-ca.crt\""
Oct 30 22:23:36.384707 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.384684    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:23:36.384836 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.384748    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-predictive-sklearn-kube-rbac-proxy-sar-config\""
Oct 30 22:23:36.384836 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.384795    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"seaweedfs-s3-creds\""
Oct 30 22:23:36.384836 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.384796    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"kube-root-ca.crt\""
Oct 30 22:23:36.385004 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.384853    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-predictive-sklearn-predictor-serving-cert\""
Oct 30 22:23:36.385004 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.384920    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"default-dockercfg-nnb5w\""
Oct 30 22:23:36.386012 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.385994    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"]
Oct 30 22:23:36.420711 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.420680    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/686192b8-37f7-406b-8060-60a99a73d6b5-kube-rbac-proxy-sar-config\") pod \"isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") " pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:36.420803 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.420717    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb25z\" (UniqueName: \"kubernetes.io/projected/686192b8-37f7-406b-8060-60a99a73d6b5-kube-api-access-zb25z\") pod \"isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") " pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:36.420866 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.420845    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/686192b8-37f7-406b-8060-60a99a73d6b5-kserve-provision-location\") pod \"isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") " pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:36.420910 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.420894    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/686192b8-37f7-406b-8060-60a99a73d6b5-cabundle-cert\") pod \"isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") " pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:36.420965 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.420949    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/686192b8-37f7-406b-8060-60a99a73d6b5-proxy-tls\") pod \"isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") " pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:36.421256 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.421238    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/686192b8-37f7-406b-8060-60a99a73d6b5-kserve-provision-location\") pod \"isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") " pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:36.421308 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.421291    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/686192b8-37f7-406b-8060-60a99a73d6b5-kube-rbac-proxy-sar-config\") pod \"isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") " pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:36.421439 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.421422    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/686192b8-37f7-406b-8060-60a99a73d6b5-cabundle-cert\") pod \"isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") " pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:36.423417 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.423401    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/686192b8-37f7-406b-8060-60a99a73d6b5-proxy-tls\") pod \"isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") " pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:36.485562 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.485537    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb25z\" (UniqueName: \"kubernetes.io/projected/686192b8-37f7-406b-8060-60a99a73d6b5-kube-api-access-zb25z\") pod \"isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") " pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:36.592169 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:36.592116    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:23:37.014277 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:37.014230    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"]
Oct 30 22:23:37.017959 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:23:37.017930    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod686192b8_37f7_406b_8060_60a99a73d6b5.slice/crio-41d885220fda589b4d7aa2375614d02349e719355901e2c124ea6ffe5c05536d WatchSource:0}: Error finding container 41d885220fda589b4d7aa2375614d02349e719355901e2c124ea6ffe5c05536d: Status 404 returned error can't find the container with id 41d885220fda589b4d7aa2375614d02349e719355901e2c124ea6ffe5c05536d
Oct 30 22:23:37.283954 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:37.283860    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" event={"ID":"686192b8-37f7-406b-8060-60a99a73d6b5","Type":"ContainerStarted","Data":"65b1c5ac62228c9f55fc5e5649bbcd877a52385b25f2025f0f076d8927f57a04"}
Oct 30 22:23:37.283954 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:37.283937    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" event={"ID":"686192b8-37f7-406b-8060-60a99a73d6b5","Type":"ContainerStarted","Data":"41d885220fda589b4d7aa2375614d02349e719355901e2c124ea6ffe5c05536d"}
Oct 30 22:23:41.301333 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:41.301303    5896 generic.go:356] "Generic (PLEG): container finished" podID="686192b8-37f7-406b-8060-60a99a73d6b5" containerID="65b1c5ac62228c9f55fc5e5649bbcd877a52385b25f2025f0f076d8927f57a04" exitCode=0
Oct 30 22:23:41.301724 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:41.301375    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" event={"ID":"686192b8-37f7-406b-8060-60a99a73d6b5","Type":"ContainerDied","Data":"65b1c5ac62228c9f55fc5e5649bbcd877a52385b25f2025f0f076d8927f57a04"}
Oct 30 22:23:41.302788 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:23:41.302772    5896 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:24:02.630703 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:24:02.630665    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:24:04.414448 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:24:04.414379    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" event={"ID":"686192b8-37f7-406b-8060-60a99a73d6b5","Type":"ContainerStarted","Data":"c7d7cdc68403830301cd69a15bc819fe01098bbfddda13f1f342405e9d6b15e2"}
Oct 30 22:24:04.414865 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:24:04.414450    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" event={"ID":"686192b8-37f7-406b-8060-60a99a73d6b5","Type":"ContainerStarted","Data":"7c0952462d32fa8b7eb42c6729a89c04349badd6a5173bd5083c3e0a4a371be6"}
Oct 30 22:24:04.414865 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:24:04.414682    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:24:04.414865 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:24:04.414828    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:24:04.415923 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:24:04.415887    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:24:04.588550 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:24:04.588501    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" podStartSLOduration=1.807074788 podStartE2EDuration="28.588489509s" totalImagesPullingTime="22.665932295s" totalInitContainerRuntime="4.115482426s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:23:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:04.586229537 +0000 UTC m=+896.997695514" watchObservedRunningTime="2026-10-30 22:24:04.588489509 +0000 UTC m=+896.999955476"
Oct 30 22:24:05.418278 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:24:05.418234    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:24:07.840313 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:24:07.840290    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:24:07.848386 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:24:07.848364    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:24:10.422541 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:24:10.422512    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:24:10.423072 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:24:10.423038    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:24:20.423016 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:24:20.422925    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:24:30.423831 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:24:30.423786    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:24:32.629441 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:24:32.629403    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:24:40.423437 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:24:40.423400    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:24:50.423530 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:24:50.423488    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:25:00.423718 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:00.423676    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:25:02.626487 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:25:02.626451    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:25:10.423826 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:10.423793    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:25:16.895855 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:16.895820    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"]
Oct 30 22:25:16.896355 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:16.896061    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kserve-container" containerID="cri-o://7c0952462d32fa8b7eb42c6729a89c04349badd6a5173bd5083c3e0a4a371be6" gracePeriod=30
Oct 30 22:25:16.896355 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:16.896105    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kube-rbac-proxy" containerID="cri-o://c7d7cdc68403830301cd69a15bc819fe01098bbfddda13f1f342405e9d6b15e2" gracePeriod=30
Oct 30 22:25:17.715852 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:17.715818    5896 generic.go:356] "Generic (PLEG): container finished" podID="686192b8-37f7-406b-8060-60a99a73d6b5" containerID="c7d7cdc68403830301cd69a15bc819fe01098bbfddda13f1f342405e9d6b15e2" exitCode=0
Oct 30 22:25:17.716027 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:17.715895    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" event={"ID":"686192b8-37f7-406b-8060-60a99a73d6b5","Type":"ContainerDied","Data":"c7d7cdc68403830301cd69a15bc819fe01098bbfddda13f1f342405e9d6b15e2"}
Oct 30 22:25:20.418978 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:20.418937    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.25:8643/healthz\": dial tcp 10.134.0.25:8643: connect: connection refused"
Oct 30 22:25:20.423219 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:20.423194    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:25:21.331878 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.331856    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:25:21.454872 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.454850    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/686192b8-37f7-406b-8060-60a99a73d6b5-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/686192b8-37f7-406b-8060-60a99a73d6b5-kube-rbac-proxy-sar-config\") pod \"686192b8-37f7-406b-8060-60a99a73d6b5\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") "
Oct 30 22:25:21.455201 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.454879    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/686192b8-37f7-406b-8060-60a99a73d6b5-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/686192b8-37f7-406b-8060-60a99a73d6b5-cabundle-cert\") pod \"686192b8-37f7-406b-8060-60a99a73d6b5\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") "
Oct 30 22:25:21.455201 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.454904    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/686192b8-37f7-406b-8060-60a99a73d6b5-kube-api-access-zb25z\" (UniqueName: \"kubernetes.io/projected/686192b8-37f7-406b-8060-60a99a73d6b5-kube-api-access-zb25z\") pod \"686192b8-37f7-406b-8060-60a99a73d6b5\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") "
Oct 30 22:25:21.455201 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.454961    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/686192b8-37f7-406b-8060-60a99a73d6b5-proxy-tls\" (UniqueName: \"kubernetes.io/secret/686192b8-37f7-406b-8060-60a99a73d6b5-proxy-tls\") pod \"686192b8-37f7-406b-8060-60a99a73d6b5\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") "
Oct 30 22:25:21.455201 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.454987    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/686192b8-37f7-406b-8060-60a99a73d6b5-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/686192b8-37f7-406b-8060-60a99a73d6b5-kserve-provision-location\") pod \"686192b8-37f7-406b-8060-60a99a73d6b5\" (UID: \"686192b8-37f7-406b-8060-60a99a73d6b5\") "
Oct 30 22:25:21.455358 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.455252    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/686192b8-37f7-406b-8060-60a99a73d6b5-kube-rbac-proxy-sar-config" pod "686192b8-37f7-406b-8060-60a99a73d6b5" (UID: "686192b8-37f7-406b-8060-60a99a73d6b5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:25:21.455358 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.455241    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/686192b8-37f7-406b-8060-60a99a73d6b5-cabundle-cert" pod "686192b8-37f7-406b-8060-60a99a73d6b5" (UID: "686192b8-37f7-406b-8060-60a99a73d6b5"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:25:21.455435 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.455358    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/686192b8-37f7-406b-8060-60a99a73d6b5-kserve-provision-location" pod "686192b8-37f7-406b-8060-60a99a73d6b5" (UID: "686192b8-37f7-406b-8060-60a99a73d6b5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:21.457089 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.457064    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/686192b8-37f7-406b-8060-60a99a73d6b5-kube-api-access-zb25z" pod "686192b8-37f7-406b-8060-60a99a73d6b5" (UID: "686192b8-37f7-406b-8060-60a99a73d6b5"). InnerVolumeSpecName "kube-api-access-zb25z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:21.457186 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.457113    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/686192b8-37f7-406b-8060-60a99a73d6b5-proxy-tls" pod "686192b8-37f7-406b-8060-60a99a73d6b5" (UID: "686192b8-37f7-406b-8060-60a99a73d6b5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:25:21.555921 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.555857    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/686192b8-37f7-406b-8060-60a99a73d6b5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:25:21.555921 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.555879    5896 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/686192b8-37f7-406b-8060-60a99a73d6b5-cabundle-cert\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:25:21.555921 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.555890    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zb25z\" (UniqueName: \"kubernetes.io/projected/686192b8-37f7-406b-8060-60a99a73d6b5-kube-api-access-zb25z\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:25:21.555921 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.555899    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/686192b8-37f7-406b-8060-60a99a73d6b5-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:25:21.555921 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.555915    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/686192b8-37f7-406b-8060-60a99a73d6b5-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:25:21.732997 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.732968    5896 generic.go:356] "Generic (PLEG): container finished" podID="686192b8-37f7-406b-8060-60a99a73d6b5" containerID="7c0952462d32fa8b7eb42c6729a89c04349badd6a5173bd5083c3e0a4a371be6" exitCode=0
Oct 30 22:25:21.733140 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.733086    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"
Oct 30 22:25:21.735997 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.735971    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" event={"ID":"686192b8-37f7-406b-8060-60a99a73d6b5","Type":"ContainerDied","Data":"7c0952462d32fa8b7eb42c6729a89c04349badd6a5173bd5083c3e0a4a371be6"}
Oct 30 22:25:21.736114 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.736024    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg" event={"ID":"686192b8-37f7-406b-8060-60a99a73d6b5","Type":"ContainerDied","Data":"41d885220fda589b4d7aa2375614d02349e719355901e2c124ea6ffe5c05536d"}
Oct 30 22:25:21.736114 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.736061    5896 scope.go:122] "RemoveContainer" containerID="c7d7cdc68403830301cd69a15bc819fe01098bbfddda13f1f342405e9d6b15e2"
Oct 30 22:25:21.744862 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.744848    5896 scope.go:122] "RemoveContainer" containerID="7c0952462d32fa8b7eb42c6729a89c04349badd6a5173bd5083c3e0a4a371be6"
Oct 30 22:25:21.752096 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.752072    5896 scope.go:122] "RemoveContainer" containerID="65b1c5ac62228c9f55fc5e5649bbcd877a52385b25f2025f0f076d8927f57a04"
Oct 30 22:25:21.759185 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.759168    5896 scope.go:122] "RemoveContainer" containerID="c7d7cdc68403830301cd69a15bc819fe01098bbfddda13f1f342405e9d6b15e2"
Oct 30 22:25:21.759439 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:25:21.759418    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7d7cdc68403830301cd69a15bc819fe01098bbfddda13f1f342405e9d6b15e2\": container with ID starting with c7d7cdc68403830301cd69a15bc819fe01098bbfddda13f1f342405e9d6b15e2 not found: ID does not exist" containerID="c7d7cdc68403830301cd69a15bc819fe01098bbfddda13f1f342405e9d6b15e2"
Oct 30 22:25:21.759519 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.759445    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7d7cdc68403830301cd69a15bc819fe01098bbfddda13f1f342405e9d6b15e2"} err="failed to get container status \"c7d7cdc68403830301cd69a15bc819fe01098bbfddda13f1f342405e9d6b15e2\": rpc error: code = NotFound desc = could not find container \"c7d7cdc68403830301cd69a15bc819fe01098bbfddda13f1f342405e9d6b15e2\": container with ID starting with c7d7cdc68403830301cd69a15bc819fe01098bbfddda13f1f342405e9d6b15e2 not found: ID does not exist"
Oct 30 22:25:21.759519 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.759472    5896 scope.go:122] "RemoveContainer" containerID="7c0952462d32fa8b7eb42c6729a89c04349badd6a5173bd5083c3e0a4a371be6"
Oct 30 22:25:21.759709 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:25:21.759691    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c0952462d32fa8b7eb42c6729a89c04349badd6a5173bd5083c3e0a4a371be6\": container with ID starting with 7c0952462d32fa8b7eb42c6729a89c04349badd6a5173bd5083c3e0a4a371be6 not found: ID does not exist" containerID="7c0952462d32fa8b7eb42c6729a89c04349badd6a5173bd5083c3e0a4a371be6"
Oct 30 22:25:21.759749 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.759714    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c0952462d32fa8b7eb42c6729a89c04349badd6a5173bd5083c3e0a4a371be6"} err="failed to get container status \"7c0952462d32fa8b7eb42c6729a89c04349badd6a5173bd5083c3e0a4a371be6\": rpc error: code = NotFound desc = could not find container \"7c0952462d32fa8b7eb42c6729a89c04349badd6a5173bd5083c3e0a4a371be6\": container with ID starting with 7c0952462d32fa8b7eb42c6729a89c04349badd6a5173bd5083c3e0a4a371be6 not found: ID does not exist"
Oct 30 22:25:21.759749 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.759729    5896 scope.go:122] "RemoveContainer" containerID="65b1c5ac62228c9f55fc5e5649bbcd877a52385b25f2025f0f076d8927f57a04"
Oct 30 22:25:21.759965 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:25:21.759946    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65b1c5ac62228c9f55fc5e5649bbcd877a52385b25f2025f0f076d8927f57a04\": container with ID starting with 65b1c5ac62228c9f55fc5e5649bbcd877a52385b25f2025f0f076d8927f57a04 not found: ID does not exist" containerID="65b1c5ac62228c9f55fc5e5649bbcd877a52385b25f2025f0f076d8927f57a04"
Oct 30 22:25:21.760016 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.759973    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65b1c5ac62228c9f55fc5e5649bbcd877a52385b25f2025f0f076d8927f57a04"} err="failed to get container status \"65b1c5ac62228c9f55fc5e5649bbcd877a52385b25f2025f0f076d8927f57a04\": rpc error: code = NotFound desc = could not find container \"65b1c5ac62228c9f55fc5e5649bbcd877a52385b25f2025f0f076d8927f57a04\": container with ID starting with 65b1c5ac62228c9f55fc5e5649bbcd877a52385b25f2025f0f076d8927f57a04 not found: ID does not exist"
Oct 30 22:25:21.903613 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.903552    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"]
Oct 30 22:25:21.954236 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:21.954213    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-bc5112d610/isvc-predictive-sklearn-predictor-85f97d59c9-xbjzg"]
Oct 30 22:25:23.736196 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:25:23.736166    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" path="/var/lib/kubelet/pods/686192b8-37f7-406b-8060-60a99a73d6b5/volumes"
Oct 30 22:25:32.627783 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:25:32.627743    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:26:02.626901 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:26:02.626863    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:26:32.629852 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:26:32.629809    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:27:02.625975 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:27:02.625934    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:27:32.627063 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:27:32.626969    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:28:02.626427 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:28:02.626387    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:28:32.628376 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:28:32.628334    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:29:02.626876 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:29:02.626841    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:29:07.868986 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:29:07.868953    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:29:07.883525 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:29:07.883501    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:29:32.628404 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:29:32.628366    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:30:02.625937 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:30:02.625901    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:30:32.627950 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:30:32.627865    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:31:02.625276 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:31:02.625237    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:31:32.632751 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:31:32.632712    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:31:39.186913 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.186879    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/curl-metrics-292a8256"]
Oct 30 22:31:39.187292 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.187219    5896 cpu_manager.go:414] "containerMap: removing container" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kserve-container"
Oct 30 22:31:39.187292 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.187230    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kserve-container"
Oct 30 22:31:39.187292 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.187249    5896 cpu_manager.go:414] "containerMap: removing container" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kube-rbac-proxy"
Oct 30 22:31:39.187292 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.187254    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kube-rbac-proxy"
Oct 30 22:31:39.187292 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.187280    5896 cpu_manager.go:414] "containerMap: removing container" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="storage-initializer"
Oct 30 22:31:39.187292 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.187285    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="storage-initializer"
Oct 30 22:31:39.187525 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.187347    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kserve-container"
Oct 30 22:31:39.187525 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.187357    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="686192b8-37f7-406b-8060-60a99a73d6b5" containerName="kube-rbac-proxy"
Oct 30 22:31:39.189349 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.189334    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/curl-metrics-292a8256"
Oct 30 22:31:39.249249 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.249227    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Oct 30 22:31:39.294040 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.294001    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-vf6ms\""
Oct 30 22:31:39.294169 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.294014    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Oct 30 22:31:39.295040 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.295018    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/curl-metrics-292a8256"]
Oct 30 22:31:39.327117 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.327097    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lwzw\" (UniqueName: \"kubernetes.io/projected/fba93ee5-fb32-4f6a-aae6-4722abf10b5f-kube-api-access-4lwzw\") pod \"curl-metrics-292a8256\" (UID: \"fba93ee5-fb32-4f6a-aae6-4722abf10b5f\") " pod="kserve/curl-metrics-292a8256"
Oct 30 22:31:39.427502 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.427472    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lwzw\" (UniqueName: \"kubernetes.io/projected/fba93ee5-fb32-4f6a-aae6-4722abf10b5f-kube-api-access-4lwzw\") pod \"curl-metrics-292a8256\" (UID: \"fba93ee5-fb32-4f6a-aae6-4722abf10b5f\") " pod="kserve/curl-metrics-292a8256"
Oct 30 22:31:39.491196 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.491165    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lwzw\" (UniqueName: \"kubernetes.io/projected/fba93ee5-fb32-4f6a-aae6-4722abf10b5f-kube-api-access-4lwzw\") pod \"curl-metrics-292a8256\" (UID: \"fba93ee5-fb32-4f6a-aae6-4722abf10b5f\") " pod="kserve/curl-metrics-292a8256"
Oct 30 22:31:39.507092 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.507061    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/curl-metrics-292a8256"
Oct 30 22:31:39.874556 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.874535    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/curl-metrics-292a8256"]
Oct 30 22:31:39.877061 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:31:39.877033    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfba93ee5_fb32_4f6a_aae6_4722abf10b5f.slice/crio-cef0830d00b24c7bf4d08fbe15ef88d8d8477f3b59bf3882789805c2e89bbde8 WatchSource:0}: Error finding container cef0830d00b24c7bf4d08fbe15ef88d8d8477f3b59bf3882789805c2e89bbde8: Status 404 returned error can't find the container with id cef0830d00b24c7bf4d08fbe15ef88d8d8477f3b59bf3882789805c2e89bbde8
Oct 30 22:31:39.878987 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:39.878971    5896 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:31:40.248182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:40.248153    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/curl-metrics-292a8256" event={"ID":"fba93ee5-fb32-4f6a-aae6-4722abf10b5f","Type":"ContainerStarted","Data":"cef0830d00b24c7bf4d08fbe15ef88d8d8477f3b59bf3882789805c2e89bbde8"}
Oct 30 22:31:42.256957 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:42.256920    5896 generic.go:356] "Generic (PLEG): container finished" podID="fba93ee5-fb32-4f6a-aae6-4722abf10b5f" containerID="cb51992e27b932e61cf3a54b3ee44bc7170cbecb51f096b01209cbff8836cc0c" exitCode=0
Oct 30 22:31:42.257338 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:42.257045    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/curl-metrics-292a8256" event={"ID":"fba93ee5-fb32-4f6a-aae6-4722abf10b5f","Type":"ContainerDied","Data":"cb51992e27b932e61cf3a54b3ee44bc7170cbecb51f096b01209cbff8836cc0c"}
Oct 30 22:31:43.381291 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:43.381268    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/curl-metrics-292a8256"
Oct 30 22:31:43.459411 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:43.459387    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fba93ee5-fb32-4f6a-aae6-4722abf10b5f-kube-api-access-4lwzw\" (UniqueName: \"kubernetes.io/projected/fba93ee5-fb32-4f6a-aae6-4722abf10b5f-kube-api-access-4lwzw\") pod \"fba93ee5-fb32-4f6a-aae6-4722abf10b5f\" (UID: \"fba93ee5-fb32-4f6a-aae6-4722abf10b5f\") "
Oct 30 22:31:43.461468 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:43.461451    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fba93ee5-fb32-4f6a-aae6-4722abf10b5f-kube-api-access-4lwzw" pod "fba93ee5-fb32-4f6a-aae6-4722abf10b5f" (UID: "fba93ee5-fb32-4f6a-aae6-4722abf10b5f"). InnerVolumeSpecName "kube-api-access-4lwzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:43.560577 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:43.560516    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lwzw\" (UniqueName: \"kubernetes.io/projected/fba93ee5-fb32-4f6a-aae6-4722abf10b5f-kube-api-access-4lwzw\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:43.693286 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:43.693262    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_curl-metrics-292a8256_fba93ee5-fb32-4f6a-aae6-4722abf10b5f/curl-metrics-292a8256/0.log"
Oct 30 22:31:43.817982 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:43.817920    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kserve/curl-metrics-292a8256"]
Oct 30 22:31:43.826035 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:43.826010    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kserve/curl-metrics-292a8256"]
Oct 30 22:31:44.265813 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:44.265778    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/curl-metrics-292a8256"
Oct 30 22:31:44.265813 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:44.265805    5896 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cef0830d00b24c7bf4d08fbe15ef88d8d8477f3b59bf3882789805c2e89bbde8"
Oct 30 22:31:44.319787 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:31:44.319755    5896 status_manager.go:1045] "Failed to get status for pod" err="pods \"curl-metrics-292a8256\" is forbidden: User \"system:node:ip-10-0-138-202.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kserve\": no relationship found between node 'ip-10-0-138-202.us-west-1.compute.internal' and this object" podUID="fba93ee5-fb32-4f6a-aae6-4722abf10b5f" pod="kserve/curl-metrics-292a8256"
Oct 30 22:31:44.412598 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:44.412574    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/curl-metrics-c2aafeb9"]
Oct 30 22:31:44.412950 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:44.412918    5896 cpu_manager.go:414] "containerMap: removing container" podUID="fba93ee5-fb32-4f6a-aae6-4722abf10b5f" containerName="curl-metrics-292a8256"
Oct 30 22:31:44.412950 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:44.412929    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fba93ee5-fb32-4f6a-aae6-4722abf10b5f" containerName="curl-metrics-292a8256"
Oct 30 22:31:44.413036 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:44.412982    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="fba93ee5-fb32-4f6a-aae6-4722abf10b5f" containerName="curl-metrics-292a8256"
Oct 30 22:31:44.415861 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:44.415847    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/curl-metrics-c2aafeb9"
Oct 30 22:31:44.467571 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:44.467543    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwf8k\" (UniqueName: \"kubernetes.io/projected/dc479fb8-8b8c-4eec-b264-f49e657ebedf-kube-api-access-kwf8k\") pod \"curl-metrics-c2aafeb9\" (UID: \"dc479fb8-8b8c-4eec-b264-f49e657ebedf\") " pod="kserve/curl-metrics-c2aafeb9"
Oct 30 22:31:44.473536 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:44.473515    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/curl-metrics-c2aafeb9"]
Oct 30 22:31:44.473536 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:44.473527    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-vf6ms\""
Oct 30 22:31:44.523167 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:44.523119    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Oct 30 22:31:44.523167 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:44.523146    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Oct 30 22:31:44.568228 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:44.568207    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwf8k\" (UniqueName: \"kubernetes.io/projected/dc479fb8-8b8c-4eec-b264-f49e657ebedf-kube-api-access-kwf8k\") pod \"curl-metrics-c2aafeb9\" (UID: \"dc479fb8-8b8c-4eec-b264-f49e657ebedf\") " pod="kserve/curl-metrics-c2aafeb9"
Oct 30 22:31:44.585806 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:31:44.585779    5896 status_manager.go:1045] "Failed to get status for pod" err="pods \"curl-metrics-292a8256\" is forbidden: User \"system:node:ip-10-0-138-202.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kserve\": no relationship found between node 'ip-10-0-138-202.us-west-1.compute.internal' and this object" podUID="fba93ee5-fb32-4f6a-aae6-4722abf10b5f" pod="kserve/curl-metrics-292a8256"
Oct 30 22:31:44.632229 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:44.632207    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwf8k\" (UniqueName: \"kubernetes.io/projected/dc479fb8-8b8c-4eec-b264-f49e657ebedf-kube-api-access-kwf8k\") pod \"curl-metrics-c2aafeb9\" (UID: \"dc479fb8-8b8c-4eec-b264-f49e657ebedf\") " pod="kserve/curl-metrics-c2aafeb9"
Oct 30 22:31:44.741724 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:44.741699    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/curl-metrics-c2aafeb9"
Oct 30 22:31:45.110601 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:45.110388    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/curl-metrics-c2aafeb9"]
Oct 30 22:31:45.113486 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:31:45.113456    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc479fb8_8b8c_4eec_b264_f49e657ebedf.slice/crio-2ea57075b3894603461df5eaaaed1a9bc2587959da68ae88da2de47209d59861 WatchSource:0}: Error finding container 2ea57075b3894603461df5eaaaed1a9bc2587959da68ae88da2de47209d59861: Status 404 returned error can't find the container with id 2ea57075b3894603461df5eaaaed1a9bc2587959da68ae88da2de47209d59861
Oct 30 22:31:45.277346 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:45.277257    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/curl-metrics-c2aafeb9" event={"ID":"dc479fb8-8b8c-4eec-b264-f49e657ebedf","Type":"ContainerStarted","Data":"4aec3941bafd3df7e8fa4d2fa6de957501551ae088c1b6461479aceaa92eec67"}
Oct 30 22:31:45.277346 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:45.277334    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/curl-metrics-c2aafeb9" event={"ID":"dc479fb8-8b8c-4eec-b264-f49e657ebedf","Type":"ContainerStarted","Data":"2ea57075b3894603461df5eaaaed1a9bc2587959da68ae88da2de47209d59861"}
Oct 30 22:31:45.395845 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:45.395752    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/curl-metrics-c2aafeb9" podStartSLOduration=1.395739338 podStartE2EDuration="1.395739338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:31:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:31:45.395212707 +0000 UTC m=+1357.806678675" watchObservedRunningTime="2026-10-30 22:31:45.395739338 +0000 UTC m=+1357.807205305"
Oct 30 22:31:45.447149 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:31:45.447121    5896 status_manager.go:1045] "Failed to get status for pod" err="pods \"curl-metrics-292a8256\" is forbidden: User \"system:node:ip-10-0-138-202.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kserve\": no relationship found between node 'ip-10-0-138-202.us-west-1.compute.internal' and this object" podUID="fba93ee5-fb32-4f6a-aae6-4722abf10b5f" pod="kserve/curl-metrics-292a8256"
Oct 30 22:31:45.725374 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:45.725354    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_curl-metrics-c2aafeb9_dc479fb8-8b8c-4eec-b264-f49e657ebedf/curl-metrics-c2aafeb9/0.log"
Oct 30 22:31:45.734745 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:45.734713    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fba93ee5-fb32-4f6a-aae6-4722abf10b5f" path="/var/lib/kubelet/pods/fba93ee5-fb32-4f6a-aae6-4722abf10b5f/volumes"
Oct 30 22:31:46.283474 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:46.283431    5896 generic.go:356] "Generic (PLEG): container finished" podID="dc479fb8-8b8c-4eec-b264-f49e657ebedf" containerID="4aec3941bafd3df7e8fa4d2fa6de957501551ae088c1b6461479aceaa92eec67" exitCode=0
Oct 30 22:31:46.283629 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:46.283508    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/curl-metrics-c2aafeb9" event={"ID":"dc479fb8-8b8c-4eec-b264-f49e657ebedf","Type":"ContainerDied","Data":"4aec3941bafd3df7e8fa4d2fa6de957501551ae088c1b6461479aceaa92eec67"}
Oct 30 22:31:47.426090 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:47.426062    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/curl-metrics-c2aafeb9"
Oct 30 22:31:47.490473 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:47.490450    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc479fb8-8b8c-4eec-b264-f49e657ebedf-kube-api-access-kwf8k\" (UniqueName: \"kubernetes.io/projected/dc479fb8-8b8c-4eec-b264-f49e657ebedf-kube-api-access-kwf8k\") pod \"dc479fb8-8b8c-4eec-b264-f49e657ebedf\" (UID: \"dc479fb8-8b8c-4eec-b264-f49e657ebedf\") "
Oct 30 22:31:47.492500 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:47.492477    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc479fb8-8b8c-4eec-b264-f49e657ebedf-kube-api-access-kwf8k" pod "dc479fb8-8b8c-4eec-b264-f49e657ebedf" (UID: "dc479fb8-8b8c-4eec-b264-f49e657ebedf"). InnerVolumeSpecName "kube-api-access-kwf8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:47.584011 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:47.583943    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kserve/curl-metrics-c2aafeb9"]
Oct 30 22:31:47.591022 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:47.590998    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwf8k\" (UniqueName: \"kubernetes.io/projected/dc479fb8-8b8c-4eec-b264-f49e657ebedf-kube-api-access-kwf8k\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:47.594229 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:47.594209    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kserve/curl-metrics-c2aafeb9"]
Oct 30 22:31:47.735387 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:47.735363    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc479fb8-8b8c-4eec-b264-f49e657ebedf" path="/var/lib/kubelet/pods/dc479fb8-8b8c-4eec-b264-f49e657ebedf/volumes"
Oct 30 22:31:48.292670 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:48.292632    5896 scope.go:122] "RemoveContainer" containerID="4aec3941bafd3df7e8fa4d2fa6de957501551ae088c1b6461479aceaa92eec67"
Oct 30 22:31:48.292806 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:31:48.292674    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/curl-metrics-c2aafeb9"
Oct 30 22:32:02.627409 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:32:02.627372    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:32:32.628578 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:32:32.628544    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:33:02.626366 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:33:02.626332    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:33:32.628106 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:33:32.628032    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:34:02.626742 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:34:02.626708    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:34:07.901323 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:34:07.901298    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:34:07.911408 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:34:07.911387    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:34:32.627953 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:34:32.627918    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:35:02.625381 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:35:02.625345    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:35:32.626322 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:35:32.626287    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:36:02.626525 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:36:02.626488    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:36:32.630752 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:36:32.630673    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:37:02.625930 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:37:02.625893    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:37:32.627971 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:37:32.627934    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:38:02.626189 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:38:02.626148    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:38:07.913815 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:38:07.913764    5896 scope.go:122] "RemoveContainer" containerID="cb51992e27b932e61cf3a54b3ee44bc7170cbecb51f096b01209cbff8836cc0c"
Oct 30 22:38:32.627678 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:38:32.627621    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:39:02.625844 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:39:02.625806    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:39:07.928985 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:39:07.928955    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:39:07.941313 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:39:07.941290    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:39:32.629753 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:39:32.629632    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:40:02.625913 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:40:02.625877    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:40:32.627375 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:40:32.627341    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:41:02.625770 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:41:02.625680    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:41:32.626943 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:41:32.626904    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:42:02.626677 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:42:02.626621    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:42:14.108992 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.108962    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"]
Oct 30 22:42:14.109367 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.109258    5896 cpu_manager.go:414] "containerMap: removing container" podUID="dc479fb8-8b8c-4eec-b264-f49e657ebedf" containerName="curl-metrics-c2aafeb9"
Oct 30 22:42:14.109367 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.109267    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc479fb8-8b8c-4eec-b264-f49e657ebedf" containerName="curl-metrics-c2aafeb9"
Oct 30 22:42:14.109367 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.109337    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc479fb8-8b8c-4eec-b264-f49e657ebedf" containerName="curl-metrics-c2aafeb9"
Oct 30 22:42:14.112369 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.112352    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.202878 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.202848    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsrgd\" (UniqueName: \"kubernetes.io/projected/57282e18-8e11-4f0d-afbd-87cd0542f929-kube-api-access-rsrgd\") pod \"isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.203016 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.202889    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/57282e18-8e11-4f0d-afbd-87cd0542f929-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.203016 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.202961    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/57282e18-8e11-4f0d-afbd-87cd0542f929-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.203016 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.202997    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57282e18-8e11-4f0d-afbd-87cd0542f929-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.203121 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.203016    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/57282e18-8e11-4f0d-afbd-87cd0542f929-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.218291 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.218264    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-autogluon-v2-variants-kube-rbac-proxy-sar-config\""
Oct 30 22:42:14.218291 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.218284    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"openshift-service-ca.crt\""
Oct 30 22:42:14.218584 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.218565    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:42:14.218701 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.218589    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"seaweedfs-s3-creds\""
Oct 30 22:42:14.218701 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.218665    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"kube-root-ca.crt\""
Oct 30 22:42:14.218701 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.218687    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"default-dockercfg-s9cqw\""
Oct 30 22:42:14.218867 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.218772    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-autogluon-v2-variants-predictor-serving-cert\""
Oct 30 22:42:14.219846 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.219829    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"]
Oct 30 22:42:14.303875 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.303847    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57282e18-8e11-4f0d-afbd-87cd0542f929-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.304002 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.303882    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/57282e18-8e11-4f0d-afbd-87cd0542f929-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.304002 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.303908    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsrgd\" (UniqueName: \"kubernetes.io/projected/57282e18-8e11-4f0d-afbd-87cd0542f929-kube-api-access-rsrgd\") pod \"isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.304002 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.303949    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/57282e18-8e11-4f0d-afbd-87cd0542f929-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.304002 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.303983    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/57282e18-8e11-4f0d-afbd-87cd0542f929-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.304317 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.304295    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/57282e18-8e11-4f0d-afbd-87cd0542f929-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.304786 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.304765    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/57282e18-8e11-4f0d-afbd-87cd0542f929-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.304872 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.304765    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/57282e18-8e11-4f0d-afbd-87cd0542f929-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.306376 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.306360    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57282e18-8e11-4f0d-afbd-87cd0542f929-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.367323 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.367269    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsrgd\" (UniqueName: \"kubernetes.io/projected/57282e18-8e11-4f0d-afbd-87cd0542f929-kube-api-access-rsrgd\") pod \"isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.428329 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.428294    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:42:14.848829 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:14.848804    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"]
Oct 30 22:42:14.851242 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:42:14.851211    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57282e18_8e11_4f0d_afbd_87cd0542f929.slice/crio-0790d497e685cd9add7bd21b7a05dfefbdbf9f9108d71e7113662f8a52955863 WatchSource:0}: Error finding container 0790d497e685cd9add7bd21b7a05dfefbdbf9f9108d71e7113662f8a52955863: Status 404 returned error can't find the container with id 0790d497e685cd9add7bd21b7a05dfefbdbf9f9108d71e7113662f8a52955863
Oct 30 22:42:15.752804 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:15.752764    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp" event={"ID":"57282e18-8e11-4f0d-afbd-87cd0542f929","Type":"ContainerStarted","Data":"82343c5fd363ac7dffc25c4d5830ef26ba8abdff1f10fac2eb6dec231a5b9206"}
Oct 30 22:42:15.753245 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:15.752821    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp" event={"ID":"57282e18-8e11-4f0d-afbd-87cd0542f929","Type":"ContainerStarted","Data":"0790d497e685cd9add7bd21b7a05dfefbdbf9f9108d71e7113662f8a52955863"}
Oct 30 22:42:22.780347 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:22.780307    5896 generic.go:356] "Generic (PLEG): container finished" podID="57282e18-8e11-4f0d-afbd-87cd0542f929" containerID="82343c5fd363ac7dffc25c4d5830ef26ba8abdff1f10fac2eb6dec231a5b9206" exitCode=0
Oct 30 22:42:22.780759 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:22.780377    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp" event={"ID":"57282e18-8e11-4f0d-afbd-87cd0542f929","Type":"ContainerDied","Data":"82343c5fd363ac7dffc25c4d5830ef26ba8abdff1f10fac2eb6dec231a5b9206"}
Oct 30 22:42:22.781589 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:42:22.781572    5896 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:42:32.633370 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:42:32.633336    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:43:02.631159 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:43:02.631122    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:43:32.632189 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:43:32.632147    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:44:02.631492 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:44:02.631383    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:44:07.957995 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:44:07.957965    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:44:08.229392 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:44:07.971011    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:44:10.275766 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:44:10.275714    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp" event={"ID":"57282e18-8e11-4f0d-afbd-87cd0542f929","Type":"ContainerStarted","Data":"3f2611e96b4c68674919133d63c6878db47025217a98073364f8abe5cc80df25"}
Oct 30 22:44:10.276193 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:44:10.275806    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:44:10.276193 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:44:10.275837    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp" event={"ID":"57282e18-8e11-4f0d-afbd-87cd0542f929","Type":"ContainerStarted","Data":"28b14c342299d46d324e81b3ea928c4fd3f891d42f5108224f521a20edc4cb4d"}
Oct 30 22:44:10.449760 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:44:10.449713    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp" podStartSLOduration=2.78702827 podStartE2EDuration="1m56.449699939s" totalImagesPullingTime="1m46.692721869s" totalInitContainerRuntime="6.9699498s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:42:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:44:10.448959948 +0000 UTC m=+2102.860425928" watchObservedRunningTime="2026-10-30 22:44:10.449699939 +0000 UTC m=+2102.861165905"
Oct 30 22:44:11.281589 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:44:11.281553    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:44:17.291427 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:44:17.291397    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:44:32.628541 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:44:32.628499    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:45:02.626633 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:45:02.626598    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:45:32.629959 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:45:32.629881    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:45:46.763339 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:45:46.763299    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:45:57.771965 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:45:57.771919    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp" podUID="57282e18-8e11-4f0d-afbd-87cd0542f929" containerName="kserve-container" probeResult="failure" output="Get \"http://10.134.0.28:8080/v2/models/isvc-autogluon-v2-variants/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:46:01.427312 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:01.427279    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"]
Oct 30 22:46:01.427715 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:01.427515    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp" podUID="57282e18-8e11-4f0d-afbd-87cd0542f929" containerName="kserve-container" containerID="cri-o://28b14c342299d46d324e81b3ea928c4fd3f891d42f5108224f521a20edc4cb4d" gracePeriod=30
Oct 30 22:46:01.427715 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:01.427603    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp" podUID="57282e18-8e11-4f0d-afbd-87cd0542f929" containerName="kube-rbac-proxy" containerID="cri-o://3f2611e96b4c68674919133d63c6878db47025217a98073364f8abe5cc80df25" gracePeriod=30
Oct 30 22:46:01.734109 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:01.734073    5896 generic.go:356] "Generic (PLEG): container finished" podID="57282e18-8e11-4f0d-afbd-87cd0542f929" containerID="3f2611e96b4c68674919133d63c6878db47025217a98073364f8abe5cc80df25" exitCode=0
Oct 30 22:46:01.736572 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:01.736541    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp" event={"ID":"57282e18-8e11-4f0d-afbd-87cd0542f929","Type":"ContainerDied","Data":"3f2611e96b4c68674919133d63c6878db47025217a98073364f8abe5cc80df25"}
Oct 30 22:46:02.286333 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:02.286283    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp" podUID="57282e18-8e11-4f0d-afbd-87cd0542f929" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused"
Oct 30 22:46:02.627660 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:46:02.627563    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:46:02.744632 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:02.744606    5896 generic.go:356] "Generic (PLEG): container finished" podID="57282e18-8e11-4f0d-afbd-87cd0542f929" containerID="28b14c342299d46d324e81b3ea928c4fd3f891d42f5108224f521a20edc4cb4d" exitCode=0
Oct 30 22:46:02.744632 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:02.744681    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp" event={"ID":"57282e18-8e11-4f0d-afbd-87cd0542f929","Type":"ContainerDied","Data":"28b14c342299d46d324e81b3ea928c4fd3f891d42f5108224f521a20edc4cb4d"}
Oct 30 22:46:02.863277 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:02.863254    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:46:03.031906 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.031878    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57282e18-8e11-4f0d-afbd-87cd0542f929-kube-api-access-rsrgd\" (UniqueName: \"kubernetes.io/projected/57282e18-8e11-4f0d-afbd-87cd0542f929-kube-api-access-rsrgd\") pod \"57282e18-8e11-4f0d-afbd-87cd0542f929\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") "
Oct 30 22:46:03.032063 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.031923    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57282e18-8e11-4f0d-afbd-87cd0542f929-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/57282e18-8e11-4f0d-afbd-87cd0542f929-cabundle-cert\") pod \"57282e18-8e11-4f0d-afbd-87cd0542f929\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") "
Oct 30 22:46:03.032063 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.031990    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/57282e18-8e11-4f0d-afbd-87cd0542f929-proxy-tls\" (UniqueName: \"kubernetes.io/secret/57282e18-8e11-4f0d-afbd-87cd0542f929-proxy-tls\") pod \"57282e18-8e11-4f0d-afbd-87cd0542f929\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") "
Oct 30 22:46:03.032063 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.032010    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57282e18-8e11-4f0d-afbd-87cd0542f929-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/57282e18-8e11-4f0d-afbd-87cd0542f929-kube-rbac-proxy-sar-config\") pod \"57282e18-8e11-4f0d-afbd-87cd0542f929\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") "
Oct 30 22:46:03.032063 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.032028    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57282e18-8e11-4f0d-afbd-87cd0542f929-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/57282e18-8e11-4f0d-afbd-87cd0542f929-kserve-provision-location\") pod \"57282e18-8e11-4f0d-afbd-87cd0542f929\" (UID: \"57282e18-8e11-4f0d-afbd-87cd0542f929\") "
Oct 30 22:46:03.032373 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.032351    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57282e18-8e11-4f0d-afbd-87cd0542f929-cabundle-cert" pod "57282e18-8e11-4f0d-afbd-87cd0542f929" (UID: "57282e18-8e11-4f0d-afbd-87cd0542f929"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:46:03.032445 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.032362    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57282e18-8e11-4f0d-afbd-87cd0542f929-kube-rbac-proxy-sar-config" pod "57282e18-8e11-4f0d-afbd-87cd0542f929" (UID: "57282e18-8e11-4f0d-afbd-87cd0542f929"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:46:03.034157 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.034122    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57282e18-8e11-4f0d-afbd-87cd0542f929-proxy-tls" pod "57282e18-8e11-4f0d-afbd-87cd0542f929" (UID: "57282e18-8e11-4f0d-afbd-87cd0542f929"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:46:03.034249 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.034204    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57282e18-8e11-4f0d-afbd-87cd0542f929-kube-api-access-rsrgd" pod "57282e18-8e11-4f0d-afbd-87cd0542f929" (UID: "57282e18-8e11-4f0d-afbd-87cd0542f929"). InnerVolumeSpecName "kube-api-access-rsrgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:46:03.037962 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.037934    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57282e18-8e11-4f0d-afbd-87cd0542f929-kserve-provision-location" pod "57282e18-8e11-4f0d-afbd-87cd0542f929" (UID: "57282e18-8e11-4f0d-afbd-87cd0542f929"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:46:03.133518 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.133495    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57282e18-8e11-4f0d-afbd-87cd0542f929-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:46:03.133518 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.133518    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/57282e18-8e11-4f0d-afbd-87cd0542f929-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:46:03.133663 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.133528    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/57282e18-8e11-4f0d-afbd-87cd0542f929-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:46:03.133663 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.133537    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsrgd\" (UniqueName: \"kubernetes.io/projected/57282e18-8e11-4f0d-afbd-87cd0542f929-kube-api-access-rsrgd\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:46:03.133663 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.133546    5896 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/57282e18-8e11-4f0d-afbd-87cd0542f929-cabundle-cert\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:46:03.749798 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.749773    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"
Oct 30 22:46:03.750150 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.749781    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp" event={"ID":"57282e18-8e11-4f0d-afbd-87cd0542f929","Type":"ContainerDied","Data":"0790d497e685cd9add7bd21b7a05dfefbdbf9f9108d71e7113662f8a52955863"}
Oct 30 22:46:03.750150 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.749850    5896 scope.go:122] "RemoveContainer" containerID="3f2611e96b4c68674919133d63c6878db47025217a98073364f8abe5cc80df25"
Oct 30 22:46:03.758030 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.758008    5896 scope.go:122] "RemoveContainer" containerID="28b14c342299d46d324e81b3ea928c4fd3f891d42f5108224f521a20edc4cb4d"
Oct 30 22:46:03.765384 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.765365    5896 scope.go:122] "RemoveContainer" containerID="82343c5fd363ac7dffc25c4d5830ef26ba8abdff1f10fac2eb6dec231a5b9206"
Oct 30 22:46:03.971079 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:03.971046    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"]
Oct 30 22:46:04.021421 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:04.021345    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-959ee2f135/isvc-autogluon-v2-variants-predictor-5d6d898548-ngqkp"]
Oct 30 22:46:05.735900 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:05.735868    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57282e18-8e11-4f0d-afbd-87cd0542f929" path="/var/lib/kubelet/pods/57282e18-8e11-4f0d-afbd-87cd0542f929/volumes"
Oct 30 22:46:06.505686 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.505633    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"]
Oct 30 22:46:06.505991 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.505978    5896 cpu_manager.go:414] "containerMap: removing container" podUID="57282e18-8e11-4f0d-afbd-87cd0542f929" containerName="storage-initializer"
Oct 30 22:46:06.506041 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.505992    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57282e18-8e11-4f0d-afbd-87cd0542f929" containerName="storage-initializer"
Oct 30 22:46:06.506041 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.506005    5896 cpu_manager.go:414] "containerMap: removing container" podUID="57282e18-8e11-4f0d-afbd-87cd0542f929" containerName="kube-rbac-proxy"
Oct 30 22:46:06.506041 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.506010    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57282e18-8e11-4f0d-afbd-87cd0542f929" containerName="kube-rbac-proxy"
Oct 30 22:46:06.506041 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.506038    5896 cpu_manager.go:414] "containerMap: removing container" podUID="57282e18-8e11-4f0d-afbd-87cd0542f929" containerName="kserve-container"
Oct 30 22:46:06.506183 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.506044    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57282e18-8e11-4f0d-afbd-87cd0542f929" containerName="kserve-container"
Oct 30 22:46:06.506183 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.506099    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="57282e18-8e11-4f0d-afbd-87cd0542f929" containerName="kserve-container"
Oct 30 22:46:06.506183 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.506109    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="57282e18-8e11-4f0d-afbd-87cd0542f929" containerName="kube-rbac-proxy"
Oct 30 22:46:06.509299 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.509279    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:06.558760 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.558733    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6a9307ae-20d8-4448-b764-a31266dbed12-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:06.558877 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.558769    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a9307ae-20d8-4448-b764-a31266dbed12-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:06.558877 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.558791    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6a9307ae-20d8-4448-b764-a31266dbed12-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:06.558957 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.558922    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7xg5\" (UniqueName: \"kubernetes.io/projected/6a9307ae-20d8-4448-b764-a31266dbed12-kube-api-access-l7xg5\") pod \"isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:06.559004 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.558957    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a9307ae-20d8-4448-b764-a31266dbed12-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:06.613797 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.613768    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"kube-root-ca.crt\""
Oct 30 22:46:06.613923 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.613832    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"openshift-service-ca.crt\""
Oct 30 22:46:06.613923 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.613849    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"seaweedfs-s3-creds\""
Oct 30 22:46:06.613923 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.613849    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-autogluon-v2-noslash-kube-rbac-proxy-sar-config\""
Oct 30 22:46:06.613923 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.613871    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"default-dockercfg-s9cqw\""
Oct 30 22:46:06.613923 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.613881    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:46:06.614148 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.613950    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-autogluon-v2-noslash-predictor-serving-cert\""
Oct 30 22:46:06.615297 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.615281    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"]
Oct 30 22:46:06.659814 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.659790    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7xg5\" (UniqueName: \"kubernetes.io/projected/6a9307ae-20d8-4448-b764-a31266dbed12-kube-api-access-l7xg5\") pod \"isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:06.659895 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.659823    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a9307ae-20d8-4448-b764-a31266dbed12-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:06.659895 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.659861    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6a9307ae-20d8-4448-b764-a31266dbed12-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:06.659895 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.659889    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a9307ae-20d8-4448-b764-a31266dbed12-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:06.660054 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.659908    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6a9307ae-20d8-4448-b764-a31266dbed12-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:06.660280 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.660260    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6a9307ae-20d8-4448-b764-a31266dbed12-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:06.660568 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.660545    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6a9307ae-20d8-4448-b764-a31266dbed12-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:06.660568 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.660557    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a9307ae-20d8-4448-b764-a31266dbed12-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:06.662447 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.662427    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a9307ae-20d8-4448-b764-a31266dbed12-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:06.723498 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.723470    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7xg5\" (UniqueName: \"kubernetes.io/projected/6a9307ae-20d8-4448-b764-a31266dbed12-kube-api-access-l7xg5\") pod \"isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") " pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:06.823865 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:06.823811    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:07.244086 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:07.244054    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"]
Oct 30 22:46:07.246570 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:46:07.246530    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a9307ae_20d8_4448_b764_a31266dbed12.slice/crio-84331ef2e544341accb945c15e173d7934f169d4d2c393b77d77eea3cb277e14 WatchSource:0}: Error finding container 84331ef2e544341accb945c15e173d7934f169d4d2c393b77d77eea3cb277e14: Status 404 returned error can't find the container with id 84331ef2e544341accb945c15e173d7934f169d4d2c393b77d77eea3cb277e14
Oct 30 22:46:07.768001 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:07.767967    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7" event={"ID":"6a9307ae-20d8-4448-b764-a31266dbed12","Type":"ContainerStarted","Data":"78566b98903da20caf0fce735f3b344bc40f22d17f66fd5b4a682da2e94a1357"}
Oct 30 22:46:07.768177 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:07.768021    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7" event={"ID":"6a9307ae-20d8-4448-b764-a31266dbed12","Type":"ContainerStarted","Data":"84331ef2e544341accb945c15e173d7934f169d4d2c393b77d77eea3cb277e14"}
Oct 30 22:46:14.797866 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:14.797750    5896 generic.go:356] "Generic (PLEG): container finished" podID="6a9307ae-20d8-4448-b764-a31266dbed12" containerID="78566b98903da20caf0fce735f3b344bc40f22d17f66fd5b4a682da2e94a1357" exitCode=0
Oct 30 22:46:14.797866 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:14.797831    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7" event={"ID":"6a9307ae-20d8-4448-b764-a31266dbed12","Type":"ContainerDied","Data":"78566b98903da20caf0fce735f3b344bc40f22d17f66fd5b4a682da2e94a1357"}
Oct 30 22:46:15.803983 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:15.803947    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7" event={"ID":"6a9307ae-20d8-4448-b764-a31266dbed12","Type":"ContainerStarted","Data":"9395e7d1e92ff1cb01b051b44316ded8f0e8ffe55d4c6eb7ca33aabe795c642f"}
Oct 30 22:46:15.804458 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:15.804008    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7" event={"ID":"6a9307ae-20d8-4448-b764-a31266dbed12","Type":"ContainerStarted","Data":"4c710b21eca0f68bf4d6b39e91a218c0dbb74d813da899302016c58ddd4336d0"}
Oct 30 22:46:15.804458 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:15.804230    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:15.804458 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:15.804286    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:15.927271 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:15.927221    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7" podStartSLOduration=3.005584389 podStartE2EDuration="9.927207425s" totalImagesPullingTime="0s" totalInitContainerRuntime="6.921623036s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:46:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:46:15.926271295 +0000 UTC m=+2228.337737266" watchObservedRunningTime="2026-10-30 22:46:15.927207425 +0000 UTC m=+2228.338673388"
Oct 30 22:46:21.814370 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:46:21.814339    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:46:32.630873 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:46:32.630826    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:47:02.626612 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:47:02.626522    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:47:32.628925 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:47:32.628888    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:47:44.763361 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:47:44.763330    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:48:02.626784 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:48:02.626744    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:48:02.757894 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:02.757868    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"]
Oct 30 22:48:02.758115 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:02.758078    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7" podUID="6a9307ae-20d8-4448-b764-a31266dbed12" containerName="kserve-container" containerID="cri-o://4c710b21eca0f68bf4d6b39e91a218c0dbb74d813da899302016c58ddd4336d0" gracePeriod=30
Oct 30 22:48:02.758233 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:02.758117    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7" podUID="6a9307ae-20d8-4448-b764-a31266dbed12" containerName="kube-rbac-proxy" containerID="cri-o://9395e7d1e92ff1cb01b051b44316ded8f0e8ffe55d4c6eb7ca33aabe795c642f" gracePeriod=30
Oct 30 22:48:03.241686 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:03.241622    5896 generic.go:356] "Generic (PLEG): container finished" podID="6a9307ae-20d8-4448-b764-a31266dbed12" containerID="9395e7d1e92ff1cb01b051b44316ded8f0e8ffe55d4c6eb7ca33aabe795c642f" exitCode=0
Oct 30 22:48:03.241686 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:03.241677    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7" event={"ID":"6a9307ae-20d8-4448-b764-a31266dbed12","Type":"ContainerDied","Data":"9395e7d1e92ff1cb01b051b44316ded8f0e8ffe55d4c6eb7ca33aabe795c642f"}
Oct 30 22:48:04.097206 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.097185    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:48:04.191092 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.191063    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a9307ae-20d8-4448-b764-a31266dbed12-kube-api-access-l7xg5\" (UniqueName: \"kubernetes.io/projected/6a9307ae-20d8-4448-b764-a31266dbed12-kube-api-access-l7xg5\") pod \"6a9307ae-20d8-4448-b764-a31266dbed12\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") "
Oct 30 22:48:04.191191 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.191145    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a9307ae-20d8-4448-b764-a31266dbed12-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a9307ae-20d8-4448-b764-a31266dbed12-kube-rbac-proxy-sar-config\") pod \"6a9307ae-20d8-4448-b764-a31266dbed12\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") "
Oct 30 22:48:04.191191 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.191176    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a9307ae-20d8-4448-b764-a31266dbed12-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6a9307ae-20d8-4448-b764-a31266dbed12-kserve-provision-location\") pod \"6a9307ae-20d8-4448-b764-a31266dbed12\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") "
Oct 30 22:48:04.191260 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.191196    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6a9307ae-20d8-4448-b764-a31266dbed12-proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a9307ae-20d8-4448-b764-a31266dbed12-proxy-tls\") pod \"6a9307ae-20d8-4448-b764-a31266dbed12\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") "
Oct 30 22:48:04.191260 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.191215    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a9307ae-20d8-4448-b764-a31266dbed12-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6a9307ae-20d8-4448-b764-a31266dbed12-cabundle-cert\") pod \"6a9307ae-20d8-4448-b764-a31266dbed12\" (UID: \"6a9307ae-20d8-4448-b764-a31266dbed12\") "
Oct 30 22:48:04.191559 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.191531    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a9307ae-20d8-4448-b764-a31266dbed12-kube-rbac-proxy-sar-config" pod "6a9307ae-20d8-4448-b764-a31266dbed12" (UID: "6a9307ae-20d8-4448-b764-a31266dbed12"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:48:04.191733 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.191706    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a9307ae-20d8-4448-b764-a31266dbed12-cabundle-cert" pod "6a9307ae-20d8-4448-b764-a31266dbed12" (UID: "6a9307ae-20d8-4448-b764-a31266dbed12"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:48:04.193360 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.193332    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a9307ae-20d8-4448-b764-a31266dbed12-kube-api-access-l7xg5" pod "6a9307ae-20d8-4448-b764-a31266dbed12" (UID: "6a9307ae-20d8-4448-b764-a31266dbed12"). InnerVolumeSpecName "kube-api-access-l7xg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:48:04.193452 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.193410    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a9307ae-20d8-4448-b764-a31266dbed12-proxy-tls" pod "6a9307ae-20d8-4448-b764-a31266dbed12" (UID: "6a9307ae-20d8-4448-b764-a31266dbed12"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:48:04.195907 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.195868    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a9307ae-20d8-4448-b764-a31266dbed12-kserve-provision-location" pod "6a9307ae-20d8-4448-b764-a31266dbed12" (UID: "6a9307ae-20d8-4448-b764-a31266dbed12"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:48:04.247244 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.247216    5896 generic.go:356] "Generic (PLEG): container finished" podID="6a9307ae-20d8-4448-b764-a31266dbed12" containerID="4c710b21eca0f68bf4d6b39e91a218c0dbb74d813da899302016c58ddd4336d0" exitCode=0
Oct 30 22:48:04.247370 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.247260    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7" event={"ID":"6a9307ae-20d8-4448-b764-a31266dbed12","Type":"ContainerDied","Data":"4c710b21eca0f68bf4d6b39e91a218c0dbb74d813da899302016c58ddd4336d0"}
Oct 30 22:48:04.247370 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.247269    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"
Oct 30 22:48:04.247370 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.247320    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7" event={"ID":"6a9307ae-20d8-4448-b764-a31266dbed12","Type":"ContainerDied","Data":"84331ef2e544341accb945c15e173d7934f169d4d2c393b77d77eea3cb277e14"}
Oct 30 22:48:04.247370 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.247361    5896 scope.go:122] "RemoveContainer" containerID="9395e7d1e92ff1cb01b051b44316ded8f0e8ffe55d4c6eb7ca33aabe795c642f"
Oct 30 22:48:04.256008 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.255993    5896 scope.go:122] "RemoveContainer" containerID="4c710b21eca0f68bf4d6b39e91a218c0dbb74d813da899302016c58ddd4336d0"
Oct 30 22:48:04.263289 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.263275    5896 scope.go:122] "RemoveContainer" containerID="78566b98903da20caf0fce735f3b344bc40f22d17f66fd5b4a682da2e94a1357"
Oct 30 22:48:04.270404 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.270387    5896 scope.go:122] "RemoveContainer" containerID="9395e7d1e92ff1cb01b051b44316ded8f0e8ffe55d4c6eb7ca33aabe795c642f"
Oct 30 22:48:04.270671 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:48:04.270652    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9395e7d1e92ff1cb01b051b44316ded8f0e8ffe55d4c6eb7ca33aabe795c642f\": container with ID starting with 9395e7d1e92ff1cb01b051b44316ded8f0e8ffe55d4c6eb7ca33aabe795c642f not found: ID does not exist" containerID="9395e7d1e92ff1cb01b051b44316ded8f0e8ffe55d4c6eb7ca33aabe795c642f"
Oct 30 22:48:04.270726 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.270682    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9395e7d1e92ff1cb01b051b44316ded8f0e8ffe55d4c6eb7ca33aabe795c642f"} err="failed to get container status \"9395e7d1e92ff1cb01b051b44316ded8f0e8ffe55d4c6eb7ca33aabe795c642f\": rpc error: code = NotFound desc = could not find container \"9395e7d1e92ff1cb01b051b44316ded8f0e8ffe55d4c6eb7ca33aabe795c642f\": container with ID starting with 9395e7d1e92ff1cb01b051b44316ded8f0e8ffe55d4c6eb7ca33aabe795c642f not found: ID does not exist"
Oct 30 22:48:04.270726 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.270699    5896 scope.go:122] "RemoveContainer" containerID="4c710b21eca0f68bf4d6b39e91a218c0dbb74d813da899302016c58ddd4336d0"
Oct 30 22:48:04.270969 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:48:04.270950    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c710b21eca0f68bf4d6b39e91a218c0dbb74d813da899302016c58ddd4336d0\": container with ID starting with 4c710b21eca0f68bf4d6b39e91a218c0dbb74d813da899302016c58ddd4336d0 not found: ID does not exist" containerID="4c710b21eca0f68bf4d6b39e91a218c0dbb74d813da899302016c58ddd4336d0"
Oct 30 22:48:04.271011 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.270976    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c710b21eca0f68bf4d6b39e91a218c0dbb74d813da899302016c58ddd4336d0"} err="failed to get container status \"4c710b21eca0f68bf4d6b39e91a218c0dbb74d813da899302016c58ddd4336d0\": rpc error: code = NotFound desc = could not find container \"4c710b21eca0f68bf4d6b39e91a218c0dbb74d813da899302016c58ddd4336d0\": container with ID starting with 4c710b21eca0f68bf4d6b39e91a218c0dbb74d813da899302016c58ddd4336d0 not found: ID does not exist"
Oct 30 22:48:04.271011 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.270990    5896 scope.go:122] "RemoveContainer" containerID="78566b98903da20caf0fce735f3b344bc40f22d17f66fd5b4a682da2e94a1357"
Oct 30 22:48:04.271193 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:48:04.271179    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78566b98903da20caf0fce735f3b344bc40f22d17f66fd5b4a682da2e94a1357\": container with ID starting with 78566b98903da20caf0fce735f3b344bc40f22d17f66fd5b4a682da2e94a1357 not found: ID does not exist" containerID="78566b98903da20caf0fce735f3b344bc40f22d17f66fd5b4a682da2e94a1357"
Oct 30 22:48:04.271239 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.271197    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78566b98903da20caf0fce735f3b344bc40f22d17f66fd5b4a682da2e94a1357"} err="failed to get container status \"78566b98903da20caf0fce735f3b344bc40f22d17f66fd5b4a682da2e94a1357\": rpc error: code = NotFound desc = could not find container \"78566b98903da20caf0fce735f3b344bc40f22d17f66fd5b4a682da2e94a1357\": container with ID starting with 78566b98903da20caf0fce735f3b344bc40f22d17f66fd5b4a682da2e94a1357 not found: ID does not exist"
Oct 30 22:48:04.292220 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.292203    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7xg5\" (UniqueName: \"kubernetes.io/projected/6a9307ae-20d8-4448-b764-a31266dbed12-kube-api-access-l7xg5\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:48:04.292292 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.292223    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a9307ae-20d8-4448-b764-a31266dbed12-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:48:04.292292 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.292233    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6a9307ae-20d8-4448-b764-a31266dbed12-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:48:04.292292 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.292241    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a9307ae-20d8-4448-b764-a31266dbed12-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:48:04.292292 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.292250    5896 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6a9307ae-20d8-4448-b764-a31266dbed12-cabundle-cert\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:48:04.417157 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.417128    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"]
Oct 30 22:48:04.423705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:04.423684    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-959ee2f135/isvc-autogluon-v2-noslash-predictor-54b567cd77-bppp7"]
Oct 30 22:48:05.735149 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:48:05.735113    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a9307ae-20d8-4448-b764-a31266dbed12" path="/var/lib/kubelet/pods/6a9307ae-20d8-4448-b764-a31266dbed12/volumes"
Oct 30 22:48:32.629689 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:48:32.629595    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:49:02.625966 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:49:02.625930    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:49:07.986019 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:07.985984    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:49:07.998128 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:07.998108    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:49:32.628107 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:49:32.628068    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:49:55.642489 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.642451    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"]
Oct 30 22:49:55.643052 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.642942    5896 cpu_manager.go:414] "containerMap: removing container" podUID="6a9307ae-20d8-4448-b764-a31266dbed12" containerName="storage-initializer"
Oct 30 22:49:55.643052 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.642961    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a9307ae-20d8-4448-b764-a31266dbed12" containerName="storage-initializer"
Oct 30 22:49:55.643052 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.642985    5896 cpu_manager.go:414] "containerMap: removing container" podUID="6a9307ae-20d8-4448-b764-a31266dbed12" containerName="kserve-container"
Oct 30 22:49:55.643052 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.642993    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a9307ae-20d8-4448-b764-a31266dbed12" containerName="kserve-container"
Oct 30 22:49:55.643052 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.643028    5896 cpu_manager.go:414] "containerMap: removing container" podUID="6a9307ae-20d8-4448-b764-a31266dbed12" containerName="kube-rbac-proxy"
Oct 30 22:49:55.643052 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.643035    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a9307ae-20d8-4448-b764-a31266dbed12" containerName="kube-rbac-proxy"
Oct 30 22:49:55.643373 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.643115    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a9307ae-20d8-4448-b764-a31266dbed12" containerName="kube-rbac-proxy"
Oct 30 22:49:55.643373 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.643133    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a9307ae-20d8-4448-b764-a31266dbed12" containerName="kserve-container"
Oct 30 22:49:55.646472 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.646450    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:55.701330 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.701313    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"openshift-service-ca.crt\""
Oct 30 22:49:55.726659 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.726615    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebc237c6-c481-4e73-81ae-267abd7b0372-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-7d65b89964-wnjzv\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") " pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:55.726799 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.726711    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebc237c6-c481-4e73-81ae-267abd7b0372-proxy-tls\") pod \"isvc-canary-create-predictor-7d65b89964-wnjzv\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") " pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:55.726799 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.726758    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebc237c6-c481-4e73-81ae-267abd7b0372-cabundle-cert\") pod \"isvc-canary-create-predictor-7d65b89964-wnjzv\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") " pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:55.726878 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.726806    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpvs4\" (UniqueName: \"kubernetes.io/projected/ebc237c6-c481-4e73-81ae-267abd7b0372-kube-api-access-wpvs4\") pod \"isvc-canary-create-predictor-7d65b89964-wnjzv\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") " pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:55.726878 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.726828    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebc237c6-c481-4e73-81ae-267abd7b0372-kserve-provision-location\") pod \"isvc-canary-create-predictor-7d65b89964-wnjzv\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") " pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:55.751206 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.751180    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"seaweedfs-s3-creds\""
Oct 30 22:49:55.751206 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.751196    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:49:55.751206 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.751197    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-canary-create-predictor-serving-cert\""
Oct 30 22:49:55.751417 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.751181    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-canary-create-kube-rbac-proxy-sar-config\""
Oct 30 22:49:55.751417 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.751215    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"kube-root-ca.crt\""
Oct 30 22:49:55.751417 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.751253    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"default-dockercfg-s9cqw\""
Oct 30 22:49:55.752521 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.752504    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"]
Oct 30 22:49:55.828100 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.828078    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebc237c6-c481-4e73-81ae-267abd7b0372-proxy-tls\") pod \"isvc-canary-create-predictor-7d65b89964-wnjzv\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") " pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:55.828203 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.828120    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebc237c6-c481-4e73-81ae-267abd7b0372-cabundle-cert\") pod \"isvc-canary-create-predictor-7d65b89964-wnjzv\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") " pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:55.828203 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.828141    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpvs4\" (UniqueName: \"kubernetes.io/projected/ebc237c6-c481-4e73-81ae-267abd7b0372-kube-api-access-wpvs4\") pod \"isvc-canary-create-predictor-7d65b89964-wnjzv\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") " pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:55.828203 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.828163    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebc237c6-c481-4e73-81ae-267abd7b0372-kserve-provision-location\") pod \"isvc-canary-create-predictor-7d65b89964-wnjzv\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") " pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:55.828203 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.828197    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebc237c6-c481-4e73-81ae-267abd7b0372-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-7d65b89964-wnjzv\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") " pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:55.828593 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.828576    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebc237c6-c481-4e73-81ae-267abd7b0372-kserve-provision-location\") pod \"isvc-canary-create-predictor-7d65b89964-wnjzv\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") " pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:55.828811 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.828794    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebc237c6-c481-4e73-81ae-267abd7b0372-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-7d65b89964-wnjzv\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") " pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:55.828871 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.828854    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebc237c6-c481-4e73-81ae-267abd7b0372-cabundle-cert\") pod \"isvc-canary-create-predictor-7d65b89964-wnjzv\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") " pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:55.830699 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.830683    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebc237c6-c481-4e73-81ae-267abd7b0372-proxy-tls\") pod \"isvc-canary-create-predictor-7d65b89964-wnjzv\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") " pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:55.892251 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.892224    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpvs4\" (UniqueName: \"kubernetes.io/projected/ebc237c6-c481-4e73-81ae-267abd7b0372-kube-api-access-wpvs4\") pod \"isvc-canary-create-predictor-7d65b89964-wnjzv\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") " pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:55.963194 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:55.963176    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:49:56.384173 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:56.384145    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"]
Oct 30 22:49:56.386804 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:49:56.386761    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebc237c6_c481_4e73_81ae_267abd7b0372.slice/crio-ff9ead8067b9ba433632a32ef47427278f60d66ed73966f7f8a04cdde9712599 WatchSource:0}: Error finding container ff9ead8067b9ba433632a32ef47427278f60d66ed73966f7f8a04cdde9712599: Status 404 returned error can't find the container with id ff9ead8067b9ba433632a32ef47427278f60d66ed73966f7f8a04cdde9712599
Oct 30 22:49:56.702678 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:56.702566    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" event={"ID":"ebc237c6-c481-4e73-81ae-267abd7b0372","Type":"ContainerStarted","Data":"940f35e9d459ca7c818005c412a8fd990ac3ea2a4a4d1bf15beda54affedcb56"}
Oct 30 22:49:56.702678 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:49:56.702621    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" event={"ID":"ebc237c6-c481-4e73-81ae-267abd7b0372","Type":"ContainerStarted","Data":"ff9ead8067b9ba433632a32ef47427278f60d66ed73966f7f8a04cdde9712599"}
Oct 30 22:50:00.720230 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:50:00.720200    5896 generic.go:356] "Generic (PLEG): container finished" podID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerID="940f35e9d459ca7c818005c412a8fd990ac3ea2a4a4d1bf15beda54affedcb56" exitCode=0
Oct 30 22:50:00.720542 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:50:00.720236    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" event={"ID":"ebc237c6-c481-4e73-81ae-267abd7b0372","Type":"ContainerDied","Data":"940f35e9d459ca7c818005c412a8fd990ac3ea2a4a4d1bf15beda54affedcb56"}
Oct 30 22:50:00.721432 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:50:00.721417    5896 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:50:02.641947 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:50:02.641915    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:50:15.795751 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:50:15.795715    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" event={"ID":"ebc237c6-c481-4e73-81ae-267abd7b0372","Type":"ContainerStarted","Data":"3705bd08d8b3aecf9e93bbfdeedbdee4d6b5458ca12600f4ca2bc4e6baa47e6c"}
Oct 30 22:50:15.796129 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:50:15.795764    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" event={"ID":"ebc237c6-c481-4e73-81ae-267abd7b0372","Type":"ContainerStarted","Data":"ba795815f3d13e1931a1509f85041ea88366e08cf310af2fb456225ce6d2fd25"}
Oct 30 22:50:15.796129 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:50:15.796034    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:50:16.004290 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:50:16.004243    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" podStartSLOduration=2.618775555 podStartE2EDuration="21.004231218s" totalImagesPullingTime="14.193203928s" totalInitContainerRuntime="4.192251735s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:49:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:50:15.971298542 +0000 UTC m=+2468.382764533" watchObservedRunningTime="2026-10-30 22:50:16.004231218 +0000 UTC m=+2468.415697185"
Oct 30 22:50:16.799754 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:50:16.799723    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:50:16.800780 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:50:16.800752    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:50:17.802893 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:50:17.802854    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:50:22.807860 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:50:22.807829    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:50:22.808371 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:50:22.808347    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:50:32.627978 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:50:32.627937    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:50:32.809034 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:50:32.808995    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:50:42.809249 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:50:42.809211    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:50:52.808296 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:50:52.808251    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:51:02.626371 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:51:02.626332    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:51:02.808945 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:02.808903    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:51:12.809128 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:12.809083    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:51:22.809049 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:22.808962    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:51:28.731781 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:28.731752    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:51:32.627578 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:51:32.627537    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:51:34.770184 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:34.770131    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"]
Oct 30 22:51:34.770700 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:34.770433    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kserve-container" containerID="cri-o://ba795815f3d13e1931a1509f85041ea88366e08cf310af2fb456225ce6d2fd25" gracePeriod=30
Oct 30 22:51:34.770700 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:34.770478    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kube-rbac-proxy" containerID="cri-o://3705bd08d8b3aecf9e93bbfdeedbdee4d6b5458ca12600f4ca2bc4e6baa47e6c" gracePeriod=30
Oct 30 22:51:35.110742 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:35.110661    5896 generic.go:356] "Generic (PLEG): container finished" podID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerID="3705bd08d8b3aecf9e93bbfdeedbdee4d6b5458ca12600f4ca2bc4e6baa47e6c" exitCode=0
Oct 30 22:51:35.110742 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:35.110710    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" event={"ID":"ebc237c6-c481-4e73-81ae-267abd7b0372","Type":"ContainerDied","Data":"3705bd08d8b3aecf9e93bbfdeedbdee4d6b5458ca12600f4ca2bc4e6baa47e6c"}
Oct 30 22:51:37.803895 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:37.803855    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.30:8643/healthz\": dial tcp 10.134.0.30:8643: connect: connection refused"
Oct 30 22:51:38.731289 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:38.731252    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:51:39.129378 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.129347    5896 generic.go:356] "Generic (PLEG): container finished" podID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerID="ba795815f3d13e1931a1509f85041ea88366e08cf310af2fb456225ce6d2fd25" exitCode=0
Oct 30 22:51:39.129753 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.129385    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" event={"ID":"ebc237c6-c481-4e73-81ae-267abd7b0372","Type":"ContainerDied","Data":"ba795815f3d13e1931a1509f85041ea88366e08cf310af2fb456225ce6d2fd25"}
Oct 30 22:51:39.216186 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.216164    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:51:39.252470 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.252447    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebc237c6-c481-4e73-81ae-267abd7b0372-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebc237c6-c481-4e73-81ae-267abd7b0372-kserve-provision-location\") pod \"ebc237c6-c481-4e73-81ae-267abd7b0372\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") "
Oct 30 22:51:39.252590 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.252498    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebc237c6-c481-4e73-81ae-267abd7b0372-kube-api-access-wpvs4\" (UniqueName: \"kubernetes.io/projected/ebc237c6-c481-4e73-81ae-267abd7b0372-kube-api-access-wpvs4\") pod \"ebc237c6-c481-4e73-81ae-267abd7b0372\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") "
Oct 30 22:51:39.252590 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.252523    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ebc237c6-c481-4e73-81ae-267abd7b0372-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebc237c6-c481-4e73-81ae-267abd7b0372-kube-rbac-proxy-sar-config\") pod \"ebc237c6-c481-4e73-81ae-267abd7b0372\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") "
Oct 30 22:51:39.252731 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.252663    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ebc237c6-c481-4e73-81ae-267abd7b0372-proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebc237c6-c481-4e73-81ae-267abd7b0372-proxy-tls\") pod \"ebc237c6-c481-4e73-81ae-267abd7b0372\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") "
Oct 30 22:51:39.252731 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.252711    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ebc237c6-c481-4e73-81ae-267abd7b0372-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebc237c6-c481-4e73-81ae-267abd7b0372-cabundle-cert\") pod \"ebc237c6-c481-4e73-81ae-267abd7b0372\" (UID: \"ebc237c6-c481-4e73-81ae-267abd7b0372\") "
Oct 30 22:51:39.252871 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.252846    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebc237c6-c481-4e73-81ae-267abd7b0372-kserve-provision-location" pod "ebc237c6-c481-4e73-81ae-267abd7b0372" (UID: "ebc237c6-c481-4e73-81ae-267abd7b0372"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:39.252871 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.252860    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebc237c6-c481-4e73-81ae-267abd7b0372-kube-rbac-proxy-sar-config" pod "ebc237c6-c481-4e73-81ae-267abd7b0372" (UID: "ebc237c6-c481-4e73-81ae-267abd7b0372"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:51:39.253007 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.252987    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebc237c6-c481-4e73-81ae-267abd7b0372-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:51:39.253047 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.253014    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebc237c6-c481-4e73-81ae-267abd7b0372-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:51:39.253081 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.253055    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebc237c6-c481-4e73-81ae-267abd7b0372-cabundle-cert" pod "ebc237c6-c481-4e73-81ae-267abd7b0372" (UID: "ebc237c6-c481-4e73-81ae-267abd7b0372"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:51:39.254684 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.254662    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebc237c6-c481-4e73-81ae-267abd7b0372-kube-api-access-wpvs4" pod "ebc237c6-c481-4e73-81ae-267abd7b0372" (UID: "ebc237c6-c481-4e73-81ae-267abd7b0372"). InnerVolumeSpecName "kube-api-access-wpvs4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:51:39.254849 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.254828    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebc237c6-c481-4e73-81ae-267abd7b0372-proxy-tls" pod "ebc237c6-c481-4e73-81ae-267abd7b0372" (UID: "ebc237c6-c481-4e73-81ae-267abd7b0372"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:51:39.353665 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.353593    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpvs4\" (UniqueName: \"kubernetes.io/projected/ebc237c6-c481-4e73-81ae-267abd7b0372-kube-api-access-wpvs4\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:51:39.353665 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.353612    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebc237c6-c481-4e73-81ae-267abd7b0372-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:51:39.353665 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:39.353621    5896 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebc237c6-c481-4e73-81ae-267abd7b0372-cabundle-cert\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:51:40.136608 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:40.136529    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv" event={"ID":"ebc237c6-c481-4e73-81ae-267abd7b0372","Type":"ContainerDied","Data":"ff9ead8067b9ba433632a32ef47427278f60d66ed73966f7f8a04cdde9712599"}
Oct 30 22:51:40.136608 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:40.136554    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"
Oct 30 22:51:40.136608 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:40.136596    5896 scope.go:122] "RemoveContainer" containerID="3705bd08d8b3aecf9e93bbfdeedbdee4d6b5458ca12600f4ca2bc4e6baa47e6c"
Oct 30 22:51:40.145118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:40.145096    5896 scope.go:122] "RemoveContainer" containerID="ba795815f3d13e1931a1509f85041ea88366e08cf310af2fb456225ce6d2fd25"
Oct 30 22:51:40.152450 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:40.152430    5896 scope.go:122] "RemoveContainer" containerID="940f35e9d459ca7c818005c412a8fd990ac3ea2a4a4d1bf15beda54affedcb56"
Oct 30 22:51:40.357210 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:40.357184    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"]
Oct 30 22:51:40.407496 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:40.407436    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-create-predictor-7d65b89964-wnjzv"]
Oct 30 22:51:41.735014 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:41.734979    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" path="/var/lib/kubelet/pods/ebc237c6-c481-4e73-81ae-267abd7b0372/volumes"
Oct 30 22:51:44.382905 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.382870    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"]
Oct 30 22:51:44.383295 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.383214    5896 cpu_manager.go:414] "containerMap: removing container" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kube-rbac-proxy"
Oct 30 22:51:44.383295 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.383225    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kube-rbac-proxy"
Oct 30 22:51:44.383295 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.383252    5896 cpu_manager.go:414] "containerMap: removing container" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="storage-initializer"
Oct 30 22:51:44.383295 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.383257    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="storage-initializer"
Oct 30 22:51:44.383295 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.383271    5896 cpu_manager.go:414] "containerMap: removing container" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kserve-container"
Oct 30 22:51:44.383295 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.383276    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kserve-container"
Oct 30 22:51:44.383475 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.383326    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kserve-container"
Oct 30 22:51:44.383475 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.383334    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebc237c6-c481-4e73-81ae-267abd7b0372" containerName="kube-rbac-proxy"
Oct 30 22:51:44.386676 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.386385    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:44.441687 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.441633    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"seaweedfs-s3-creds\""
Oct 30 22:51:44.491380 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.491344    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"default-dockercfg-s9cqw\""
Oct 30 22:51:44.491524 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.491404    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"openshift-service-ca.crt\""
Oct 30 22:51:44.491524 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.491417    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:51:44.491524 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.491432    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\""
Oct 30 22:51:44.491524 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.491424    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-canary-promote-predictor-serving-cert\""
Oct 30 22:51:44.491524 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.491493    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"kube-root-ca.crt\""
Oct 30 22:51:44.493270 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.493248    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"]
Oct 30 22:51:44.493551 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.493534    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/07cc7141-610c-4b63-a1a7-0df7af34437c-cabundle-cert\") pod \"isvc-canary-promote-predictor-66d68c5b56-gshbx\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:44.493587 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.493564    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nqm9\" (UniqueName: \"kubernetes.io/projected/07cc7141-610c-4b63-a1a7-0df7af34437c-kube-api-access-8nqm9\") pod \"isvc-canary-promote-predictor-66d68c5b56-gshbx\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:44.493633 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.493600    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/07cc7141-610c-4b63-a1a7-0df7af34437c-proxy-tls\") pod \"isvc-canary-promote-predictor-66d68c5b56-gshbx\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:44.493789 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.493770    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07cc7141-610c-4b63-a1a7-0df7af34437c-kserve-provision-location\") pod \"isvc-canary-promote-predictor-66d68c5b56-gshbx\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:44.493827 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.493813    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/07cc7141-610c-4b63-a1a7-0df7af34437c-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-66d68c5b56-gshbx\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:44.594552 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.594526    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nqm9\" (UniqueName: \"kubernetes.io/projected/07cc7141-610c-4b63-a1a7-0df7af34437c-kube-api-access-8nqm9\") pod \"isvc-canary-promote-predictor-66d68c5b56-gshbx\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:44.594695 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.594566    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/07cc7141-610c-4b63-a1a7-0df7af34437c-proxy-tls\") pod \"isvc-canary-promote-predictor-66d68c5b56-gshbx\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:44.594695 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.594589    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07cc7141-610c-4b63-a1a7-0df7af34437c-kserve-provision-location\") pod \"isvc-canary-promote-predictor-66d68c5b56-gshbx\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:44.594695 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.594615    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/07cc7141-610c-4b63-a1a7-0df7af34437c-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-66d68c5b56-gshbx\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:44.594695 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.594678    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/07cc7141-610c-4b63-a1a7-0df7af34437c-cabundle-cert\") pod \"isvc-canary-promote-predictor-66d68c5b56-gshbx\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:44.594997 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.594976    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07cc7141-610c-4b63-a1a7-0df7af34437c-kserve-provision-location\") pod \"isvc-canary-promote-predictor-66d68c5b56-gshbx\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:44.595318 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.595294    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/07cc7141-610c-4b63-a1a7-0df7af34437c-cabundle-cert\") pod \"isvc-canary-promote-predictor-66d68c5b56-gshbx\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:44.595434 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.595416    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/07cc7141-610c-4b63-a1a7-0df7af34437c-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-66d68c5b56-gshbx\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:44.597131 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.597114    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/07cc7141-610c-4b63-a1a7-0df7af34437c-proxy-tls\") pod \"isvc-canary-promote-predictor-66d68c5b56-gshbx\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:44.658012 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.657951    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nqm9\" (UniqueName: \"kubernetes.io/projected/07cc7141-610c-4b63-a1a7-0df7af34437c-kube-api-access-8nqm9\") pod \"isvc-canary-promote-predictor-66d68c5b56-gshbx\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:44.702118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:44.702092    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:45.127031 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:45.127008    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"]
Oct 30 22:51:45.130021 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:51:45.129990    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07cc7141_610c_4b63_a1a7_0df7af34437c.slice/crio-fb1d9513c71b65d3949ae79d4f53a6db34b074ba451f2bcf09c3da5a5e21f4bf WatchSource:0}: Error finding container fb1d9513c71b65d3949ae79d4f53a6db34b074ba451f2bcf09c3da5a5e21f4bf: Status 404 returned error can't find the container with id fb1d9513c71b65d3949ae79d4f53a6db34b074ba451f2bcf09c3da5a5e21f4bf
Oct 30 22:51:45.160506 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:45.160484    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" event={"ID":"07cc7141-610c-4b63-a1a7-0df7af34437c","Type":"ContainerStarted","Data":"fb1d9513c71b65d3949ae79d4f53a6db34b074ba451f2bcf09c3da5a5e21f4bf"}
Oct 30 22:51:46.166550 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:46.166513    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" event={"ID":"07cc7141-610c-4b63-a1a7-0df7af34437c","Type":"ContainerStarted","Data":"22eb06e0b8e6c5dec9177f66bd759639bc023b1e4f9af17cf967725b6e438e83"}
Oct 30 22:51:49.180482 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:49.180452    5896 generic.go:356] "Generic (PLEG): container finished" podID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerID="22eb06e0b8e6c5dec9177f66bd759639bc023b1e4f9af17cf967725b6e438e83" exitCode=0
Oct 30 22:51:49.180809 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:49.180489    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" event={"ID":"07cc7141-610c-4b63-a1a7-0df7af34437c","Type":"ContainerDied","Data":"22eb06e0b8e6c5dec9177f66bd759639bc023b1e4f9af17cf967725b6e438e83"}
Oct 30 22:51:50.186258 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:50.186223    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" event={"ID":"07cc7141-610c-4b63-a1a7-0df7af34437c","Type":"ContainerStarted","Data":"3cfef3098ea501c2d962556a1e6fd24a06eb61f13448aa5baf6331c5abebdbda"}
Oct 30 22:51:50.186714 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:50.186284    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" event={"ID":"07cc7141-610c-4b63-a1a7-0df7af34437c","Type":"ContainerStarted","Data":"8318c29716e1228660fdcc7fe1bae8af6d5f163d0525181d5dbe0660e08f5969"}
Oct 30 22:51:50.186714 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:50.186556    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:50.186714 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:50.186708    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:50.187956 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:50.187922    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:51:50.309612 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:50.309568    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" podStartSLOduration=2.352601697 podStartE2EDuration="6.309553171s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.956951474s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:51:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:51:50.308958069 +0000 UTC m=+2562.720424037" watchObservedRunningTime="2026-10-30 22:51:50.309553171 +0000 UTC m=+2562.721019139"
Oct 30 22:51:51.190751 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:51.190714    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:51:56.195287 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:56.195257    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:51:56.195834 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:51:56.195807    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:52:02.626423 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:52:02.626382    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:52:06.196534 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:52:06.196502    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:52:16.196093 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:52:16.196053    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:52:26.196020 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:52:26.195981    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:52:32.627730 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:52:32.627690    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:52:36.196615 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:52:36.196578    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:52:46.195995 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:52:46.195912    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:52:56.195754 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:52:56.195716    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:53:02.625933 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:53:02.625897    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:53:05.735994 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:05.735965    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:53:12.516276 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:12.516243    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"]
Oct 30 22:53:12.516736 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:12.516472    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kserve-container" containerID="cri-o://8318c29716e1228660fdcc7fe1bae8af6d5f163d0525181d5dbe0660e08f5969" gracePeriod=30
Oct 30 22:53:12.516736 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:12.516526    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kube-rbac-proxy" containerID="cri-o://3cfef3098ea501c2d962556a1e6fd24a06eb61f13448aa5baf6331c5abebdbda" gracePeriod=30
Oct 30 22:53:13.529740 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:13.529710    5896 generic.go:356] "Generic (PLEG): container finished" podID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerID="3cfef3098ea501c2d962556a1e6fd24a06eb61f13448aa5baf6331c5abebdbda" exitCode=0
Oct 30 22:53:13.530133 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:13.529768    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" event={"ID":"07cc7141-610c-4b63-a1a7-0df7af34437c","Type":"ContainerDied","Data":"3cfef3098ea501c2d962556a1e6fd24a06eb61f13448aa5baf6331c5abebdbda"}
Oct 30 22:53:13.884281 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:13.884192    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"]
Oct 30 22:53:13.887855 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:13.887837    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:13.942172 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:13.942149    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-canary-promote-v2-predictor-serving-cert\""
Oct 30 22:53:13.975182 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:13.975150    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-66ffb95d77-svdzh\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:13.975285 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:13.975207    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-66ffb95d77-svdzh\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:13.975329 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:13.975309    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-66ffb95d77-svdzh\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:13.975370 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:13.975335    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-66ffb95d77-svdzh\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:13.975370 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:13.975360    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb2zc\" (UniqueName: \"kubernetes.io/projected/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kube-api-access-gb2zc\") pod \"isvc-canary-promote-v2-predictor-66ffb95d77-svdzh\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:13.993102 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:13.993073    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"]
Oct 30 22:53:14.076429 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:14.076387    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-66ffb95d77-svdzh\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:14.076543 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:14.076445    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-66ffb95d77-svdzh\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:14.076543 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:14.076478    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-66ffb95d77-svdzh\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:14.076543 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:14.076496    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-66ffb95d77-svdzh\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:14.076543 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:14.076511    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb2zc\" (UniqueName: \"kubernetes.io/projected/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kube-api-access-gb2zc\") pod \"isvc-canary-promote-v2-predictor-66ffb95d77-svdzh\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:14.076907 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:14.076888    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-66ffb95d77-svdzh\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:14.077116 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:14.077091    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-66ffb95d77-svdzh\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:14.077116 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:14.077102    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-66ffb95d77-svdzh\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:14.079076 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:14.079052    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-66ffb95d77-svdzh\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:14.142090 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:14.142040    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb2zc\" (UniqueName: \"kubernetes.io/projected/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kube-api-access-gb2zc\") pod \"isvc-canary-promote-v2-predictor-66ffb95d77-svdzh\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") " pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:14.204321 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:14.204287    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:14.626250 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:14.626222    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"]
Oct 30 22:53:14.628428 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:53:14.628385    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac0825f3_616f_4fbc_8fb0_3c879e753c5e.slice/crio-cc669e02c5cee623ad23dc494eee56287eb477d671257968e6cfd8daba6dc24e WatchSource:0}: Error finding container cc669e02c5cee623ad23dc494eee56287eb477d671257968e6cfd8daba6dc24e: Status 404 returned error can't find the container with id cc669e02c5cee623ad23dc494eee56287eb477d671257968e6cfd8daba6dc24e
Oct 30 22:53:15.539532 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:15.539490    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" event={"ID":"ac0825f3-616f-4fbc-8fb0-3c879e753c5e","Type":"ContainerStarted","Data":"36fd890f5a9056aed8ed4edd08d55d830176cd8dcd154e495781b9f543989969"}
Oct 30 22:53:15.539754 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:15.539552    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" event={"ID":"ac0825f3-616f-4fbc-8fb0-3c879e753c5e","Type":"ContainerStarted","Data":"cc669e02c5cee623ad23dc494eee56287eb477d671257968e6cfd8daba6dc24e"}
Oct 30 22:53:15.731802 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:15.731765    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:53:16.191939 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:16.191894    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.31:8643/healthz\": dial tcp 10.134.0.31:8643: connect: connection refused"
Oct 30 22:53:16.959465 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:16.959436    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:53:17.099533 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.099476    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/07cc7141-610c-4b63-a1a7-0df7af34437c-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/07cc7141-610c-4b63-a1a7-0df7af34437c-cabundle-cert\") pod \"07cc7141-610c-4b63-a1a7-0df7af34437c\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") "
Oct 30 22:53:17.099533 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.099523    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07cc7141-610c-4b63-a1a7-0df7af34437c-proxy-tls\" (UniqueName: \"kubernetes.io/secret/07cc7141-610c-4b63-a1a7-0df7af34437c-proxy-tls\") pod \"07cc7141-610c-4b63-a1a7-0df7af34437c\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") "
Oct 30 22:53:17.099760 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.099596    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07cc7141-610c-4b63-a1a7-0df7af34437c-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07cc7141-610c-4b63-a1a7-0df7af34437c-kserve-provision-location\") pod \"07cc7141-610c-4b63-a1a7-0df7af34437c\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") "
Oct 30 22:53:17.099760 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.099621    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07cc7141-610c-4b63-a1a7-0df7af34437c-kube-api-access-8nqm9\" (UniqueName: \"kubernetes.io/projected/07cc7141-610c-4b63-a1a7-0df7af34437c-kube-api-access-8nqm9\") pod \"07cc7141-610c-4b63-a1a7-0df7af34437c\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") "
Oct 30 22:53:17.099760 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.099675    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/07cc7141-610c-4b63-a1a7-0df7af34437c-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/07cc7141-610c-4b63-a1a7-0df7af34437c-kube-rbac-proxy-sar-config\") pod \"07cc7141-610c-4b63-a1a7-0df7af34437c\" (UID: \"07cc7141-610c-4b63-a1a7-0df7af34437c\") "
Oct 30 22:53:17.099915 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.099864    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07cc7141-610c-4b63-a1a7-0df7af34437c-cabundle-cert" pod "07cc7141-610c-4b63-a1a7-0df7af34437c" (UID: "07cc7141-610c-4b63-a1a7-0df7af34437c"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:53:17.099915 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.099892    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07cc7141-610c-4b63-a1a7-0df7af34437c-kserve-provision-location" pod "07cc7141-610c-4b63-a1a7-0df7af34437c" (UID: "07cc7141-610c-4b63-a1a7-0df7af34437c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:53:17.100139 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.100115    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07cc7141-610c-4b63-a1a7-0df7af34437c-kube-rbac-proxy-sar-config" pod "07cc7141-610c-4b63-a1a7-0df7af34437c" (UID: "07cc7141-610c-4b63-a1a7-0df7af34437c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:53:17.101806 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.101784    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07cc7141-610c-4b63-a1a7-0df7af34437c-kube-api-access-8nqm9" pod "07cc7141-610c-4b63-a1a7-0df7af34437c" (UID: "07cc7141-610c-4b63-a1a7-0df7af34437c"). InnerVolumeSpecName "kube-api-access-8nqm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:53:17.101897 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.101881    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07cc7141-610c-4b63-a1a7-0df7af34437c-proxy-tls" pod "07cc7141-610c-4b63-a1a7-0df7af34437c" (UID: "07cc7141-610c-4b63-a1a7-0df7af34437c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:53:17.201124 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.201095    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07cc7141-610c-4b63-a1a7-0df7af34437c-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:53:17.201124 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.201118    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nqm9\" (UniqueName: \"kubernetes.io/projected/07cc7141-610c-4b63-a1a7-0df7af34437c-kube-api-access-8nqm9\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:53:17.201124 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.201128    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/07cc7141-610c-4b63-a1a7-0df7af34437c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:53:17.201295 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.201137    5896 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/07cc7141-610c-4b63-a1a7-0df7af34437c-cabundle-cert\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:53:17.201295 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.201145    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/07cc7141-610c-4b63-a1a7-0df7af34437c-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:53:17.551622 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.551587    5896 generic.go:356] "Generic (PLEG): container finished" podID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerID="8318c29716e1228660fdcc7fe1bae8af6d5f163d0525181d5dbe0660e08f5969" exitCode=0
Oct 30 22:53:17.551808 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.551656    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"
Oct 30 22:53:17.551808 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.551678    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" event={"ID":"07cc7141-610c-4b63-a1a7-0df7af34437c","Type":"ContainerDied","Data":"8318c29716e1228660fdcc7fe1bae8af6d5f163d0525181d5dbe0660e08f5969"}
Oct 30 22:53:17.551808 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.551763    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx" event={"ID":"07cc7141-610c-4b63-a1a7-0df7af34437c","Type":"ContainerDied","Data":"fb1d9513c71b65d3949ae79d4f53a6db34b074ba451f2bcf09c3da5a5e21f4bf"}
Oct 30 22:53:17.551808 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.551798    5896 scope.go:122] "RemoveContainer" containerID="3cfef3098ea501c2d962556a1e6fd24a06eb61f13448aa5baf6331c5abebdbda"
Oct 30 22:53:17.560944 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.560926    5896 scope.go:122] "RemoveContainer" containerID="8318c29716e1228660fdcc7fe1bae8af6d5f163d0525181d5dbe0660e08f5969"
Oct 30 22:53:17.568447 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.568428    5896 scope.go:122] "RemoveContainer" containerID="22eb06e0b8e6c5dec9177f66bd759639bc023b1e4f9af17cf967725b6e438e83"
Oct 30 22:53:17.575341 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.575324    5896 scope.go:122] "RemoveContainer" containerID="3cfef3098ea501c2d962556a1e6fd24a06eb61f13448aa5baf6331c5abebdbda"
Oct 30 22:53:17.575585 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:53:17.575570    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cfef3098ea501c2d962556a1e6fd24a06eb61f13448aa5baf6331c5abebdbda\": container with ID starting with 3cfef3098ea501c2d962556a1e6fd24a06eb61f13448aa5baf6331c5abebdbda not found: ID does not exist" containerID="3cfef3098ea501c2d962556a1e6fd24a06eb61f13448aa5baf6331c5abebdbda"
Oct 30 22:53:17.575626 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.575594    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cfef3098ea501c2d962556a1e6fd24a06eb61f13448aa5baf6331c5abebdbda"} err="failed to get container status \"3cfef3098ea501c2d962556a1e6fd24a06eb61f13448aa5baf6331c5abebdbda\": rpc error: code = NotFound desc = could not find container \"3cfef3098ea501c2d962556a1e6fd24a06eb61f13448aa5baf6331c5abebdbda\": container with ID starting with 3cfef3098ea501c2d962556a1e6fd24a06eb61f13448aa5baf6331c5abebdbda not found: ID does not exist"
Oct 30 22:53:17.575626 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.575610    5896 scope.go:122] "RemoveContainer" containerID="8318c29716e1228660fdcc7fe1bae8af6d5f163d0525181d5dbe0660e08f5969"
Oct 30 22:53:17.575866 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:53:17.575849    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8318c29716e1228660fdcc7fe1bae8af6d5f163d0525181d5dbe0660e08f5969\": container with ID starting with 8318c29716e1228660fdcc7fe1bae8af6d5f163d0525181d5dbe0660e08f5969 not found: ID does not exist" containerID="8318c29716e1228660fdcc7fe1bae8af6d5f163d0525181d5dbe0660e08f5969"
Oct 30 22:53:17.575931 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.575870    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8318c29716e1228660fdcc7fe1bae8af6d5f163d0525181d5dbe0660e08f5969"} err="failed to get container status \"8318c29716e1228660fdcc7fe1bae8af6d5f163d0525181d5dbe0660e08f5969\": rpc error: code = NotFound desc = could not find container \"8318c29716e1228660fdcc7fe1bae8af6d5f163d0525181d5dbe0660e08f5969\": container with ID starting with 8318c29716e1228660fdcc7fe1bae8af6d5f163d0525181d5dbe0660e08f5969 not found: ID does not exist"
Oct 30 22:53:17.575931 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.575884    5896 scope.go:122] "RemoveContainer" containerID="22eb06e0b8e6c5dec9177f66bd759639bc023b1e4f9af17cf967725b6e438e83"
Oct 30 22:53:17.576086 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:53:17.576071    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22eb06e0b8e6c5dec9177f66bd759639bc023b1e4f9af17cf967725b6e438e83\": container with ID starting with 22eb06e0b8e6c5dec9177f66bd759639bc023b1e4f9af17cf967725b6e438e83 not found: ID does not exist" containerID="22eb06e0b8e6c5dec9177f66bd759639bc023b1e4f9af17cf967725b6e438e83"
Oct 30 22:53:17.576124 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.576089    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22eb06e0b8e6c5dec9177f66bd759639bc023b1e4f9af17cf967725b6e438e83"} err="failed to get container status \"22eb06e0b8e6c5dec9177f66bd759639bc023b1e4f9af17cf967725b6e438e83\": rpc error: code = NotFound desc = could not find container \"22eb06e0b8e6c5dec9177f66bd759639bc023b1e4f9af17cf967725b6e438e83\": container with ID starting with 22eb06e0b8e6c5dec9177f66bd759639bc023b1e4f9af17cf967725b6e438e83 not found: ID does not exist"
Oct 30 22:53:17.720619 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.720589    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"]
Oct 30 22:53:17.771137 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:17.771111    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-promote-predictor-66d68c5b56-gshbx"]
Oct 30 22:53:19.565428 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:19.565397    5896 generic.go:356] "Generic (PLEG): container finished" podID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerID="36fd890f5a9056aed8ed4edd08d55d830176cd8dcd154e495781b9f543989969" exitCode=0
Oct 30 22:53:19.565832 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:19.565471    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" event={"ID":"ac0825f3-616f-4fbc-8fb0-3c879e753c5e","Type":"ContainerDied","Data":"36fd890f5a9056aed8ed4edd08d55d830176cd8dcd154e495781b9f543989969"}
Oct 30 22:53:19.735151 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:19.735124    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" path="/var/lib/kubelet/pods/07cc7141-610c-4b63-a1a7-0df7af34437c/volumes"
Oct 30 22:53:20.571891 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:20.571851    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" event={"ID":"ac0825f3-616f-4fbc-8fb0-3c879e753c5e","Type":"ContainerStarted","Data":"cfe8903106f717f8bd931e46df35c5eb8776bc658218e5ec47107b105d39514b"}
Oct 30 22:53:20.572317 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:20.571925    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" event={"ID":"ac0825f3-616f-4fbc-8fb0-3c879e753c5e","Type":"ContainerStarted","Data":"990b93d4b98f84e6b1f23832328b242237b096b5be5c7dfbce0c558415d3c4ab"}
Oct 30 22:53:20.572317 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:20.572162    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:20.572317 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:20.572303    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:20.573713 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:20.573687    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:53:20.697729 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:20.697688    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" podStartSLOduration=3.729031543 podStartE2EDuration="7.697676841s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.968645298s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:53:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:53:20.697493984 +0000 UTC m=+2653.108959963" watchObservedRunningTime="2026-10-30 22:53:20.697676841 +0000 UTC m=+2653.109142802"
Oct 30 22:53:21.576779 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:21.576740    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:53:26.581742 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:26.581714    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:53:26.582327 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:26.582300    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:53:32.629906 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:53:32.629869    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:53:36.582805 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:36.582770    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:53:46.583281 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:46.583242    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:53:56.582310 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:53:56.582274    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:54:02.626716 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:54:02.626677    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:54:06.582771 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:06.582733    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:54:08.014115 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:08.014020    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:54:08.025653 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:08.025623    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:54:16.583241 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:16.583149    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:54:26.582663 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:26.582601    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:54:32.628848 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:54:32.628807    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:54:36.583002 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:36.582977    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:54:44.432874 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:44.432842    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"]
Oct 30 22:54:44.433357 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:44.433089    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kserve-container" containerID="cri-o://990b93d4b98f84e6b1f23832328b242237b096b5be5c7dfbce0c558415d3c4ab" gracePeriod=30
Oct 30 22:54:44.433357 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:44.433127    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kube-rbac-proxy" containerID="cri-o://cfe8903106f717f8bd931e46df35c5eb8776bc658218e5ec47107b105d39514b" gracePeriod=30
Oct 30 22:54:44.918351 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:44.918320    5896 generic.go:356] "Generic (PLEG): container finished" podID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerID="cfe8903106f717f8bd931e46df35c5eb8776bc658218e5ec47107b105d39514b" exitCode=0
Oct 30 22:54:44.918517 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:44.918367    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" event={"ID":"ac0825f3-616f-4fbc-8fb0-3c879e753c5e","Type":"ContainerDied","Data":"cfe8903106f717f8bd931e46df35c5eb8776bc658218e5ec47107b105d39514b"}
Oct 30 22:54:46.577907 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:46.577864    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.32:8643/healthz\": dial tcp 10.134.0.32:8643: connect: connection refused"
Oct 30 22:54:46.582623 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:46.582598    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:54:48.881237 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.881214    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:54:48.901405 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.901381    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-cabundle-cert\") pod \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") "
Oct 30 22:54:48.901513 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.901420    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kube-api-access-gb2zc\" (UniqueName: \"kubernetes.io/projected/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kube-api-access-gb2zc\") pod \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") "
Oct 30 22:54:48.901513 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.901461    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kserve-provision-location\") pod \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") "
Oct 30 22:54:48.901513 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.901497    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-proxy-tls\") pod \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") "
Oct 30 22:54:48.901703 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.901516    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kube-rbac-proxy-sar-config\") pod \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\" (UID: \"ac0825f3-616f-4fbc-8fb0-3c879e753c5e\") "
Oct 30 22:54:48.901782 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.901756    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-cabundle-cert" pod "ac0825f3-616f-4fbc-8fb0-3c879e753c5e" (UID: "ac0825f3-616f-4fbc-8fb0-3c879e753c5e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:54:48.901887 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.901800    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kserve-provision-location" pod "ac0825f3-616f-4fbc-8fb0-3c879e753c5e" (UID: "ac0825f3-616f-4fbc-8fb0-3c879e753c5e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:54:48.901985 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.901964    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kube-rbac-proxy-sar-config" pod "ac0825f3-616f-4fbc-8fb0-3c879e753c5e" (UID: "ac0825f3-616f-4fbc-8fb0-3c879e753c5e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:54:48.903734 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.903713    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-proxy-tls" pod "ac0825f3-616f-4fbc-8fb0-3c879e753c5e" (UID: "ac0825f3-616f-4fbc-8fb0-3c879e753c5e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:54:48.903808 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.903756    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kube-api-access-gb2zc" pod "ac0825f3-616f-4fbc-8fb0-3c879e753c5e" (UID: "ac0825f3-616f-4fbc-8fb0-3c879e753c5e"). InnerVolumeSpecName "kube-api-access-gb2zc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:54:48.937187 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.937131    5896 generic.go:356] "Generic (PLEG): container finished" podID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerID="990b93d4b98f84e6b1f23832328b242237b096b5be5c7dfbce0c558415d3c4ab" exitCode=0
Oct 30 22:54:48.937187 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.937169    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" event={"ID":"ac0825f3-616f-4fbc-8fb0-3c879e753c5e","Type":"ContainerDied","Data":"990b93d4b98f84e6b1f23832328b242237b096b5be5c7dfbce0c558415d3c4ab"}
Oct 30 22:54:48.937341 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.937208    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"
Oct 30 22:54:48.937341 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.937216    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh" event={"ID":"ac0825f3-616f-4fbc-8fb0-3c879e753c5e","Type":"ContainerDied","Data":"cc669e02c5cee623ad23dc494eee56287eb477d671257968e6cfd8daba6dc24e"}
Oct 30 22:54:48.937341 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.937240    5896 scope.go:122] "RemoveContainer" containerID="cfe8903106f717f8bd931e46df35c5eb8776bc658218e5ec47107b105d39514b"
Oct 30 22:54:48.949179 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.949163    5896 scope.go:122] "RemoveContainer" containerID="990b93d4b98f84e6b1f23832328b242237b096b5be5c7dfbce0c558415d3c4ab"
Oct 30 22:54:48.956359 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.956341    5896 scope.go:122] "RemoveContainer" containerID="36fd890f5a9056aed8ed4edd08d55d830176cd8dcd154e495781b9f543989969"
Oct 30 22:54:48.963450 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.963437    5896 scope.go:122] "RemoveContainer" containerID="cfe8903106f717f8bd931e46df35c5eb8776bc658218e5ec47107b105d39514b"
Oct 30 22:54:48.963706 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:54:48.963687    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfe8903106f717f8bd931e46df35c5eb8776bc658218e5ec47107b105d39514b\": container with ID starting with cfe8903106f717f8bd931e46df35c5eb8776bc658218e5ec47107b105d39514b not found: ID does not exist" containerID="cfe8903106f717f8bd931e46df35c5eb8776bc658218e5ec47107b105d39514b"
Oct 30 22:54:48.963753 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.963713    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfe8903106f717f8bd931e46df35c5eb8776bc658218e5ec47107b105d39514b"} err="failed to get container status \"cfe8903106f717f8bd931e46df35c5eb8776bc658218e5ec47107b105d39514b\": rpc error: code = NotFound desc = could not find container \"cfe8903106f717f8bd931e46df35c5eb8776bc658218e5ec47107b105d39514b\": container with ID starting with cfe8903106f717f8bd931e46df35c5eb8776bc658218e5ec47107b105d39514b not found: ID does not exist"
Oct 30 22:54:48.963753 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.963732    5896 scope.go:122] "RemoveContainer" containerID="990b93d4b98f84e6b1f23832328b242237b096b5be5c7dfbce0c558415d3c4ab"
Oct 30 22:54:48.963977 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:54:48.963957    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"990b93d4b98f84e6b1f23832328b242237b096b5be5c7dfbce0c558415d3c4ab\": container with ID starting with 990b93d4b98f84e6b1f23832328b242237b096b5be5c7dfbce0c558415d3c4ab not found: ID does not exist" containerID="990b93d4b98f84e6b1f23832328b242237b096b5be5c7dfbce0c558415d3c4ab"
Oct 30 22:54:48.964037 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.963984    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"990b93d4b98f84e6b1f23832328b242237b096b5be5c7dfbce0c558415d3c4ab"} err="failed to get container status \"990b93d4b98f84e6b1f23832328b242237b096b5be5c7dfbce0c558415d3c4ab\": rpc error: code = NotFound desc = could not find container \"990b93d4b98f84e6b1f23832328b242237b096b5be5c7dfbce0c558415d3c4ab\": container with ID starting with 990b93d4b98f84e6b1f23832328b242237b096b5be5c7dfbce0c558415d3c4ab not found: ID does not exist"
Oct 30 22:54:48.964037 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.963999    5896 scope.go:122] "RemoveContainer" containerID="36fd890f5a9056aed8ed4edd08d55d830176cd8dcd154e495781b9f543989969"
Oct 30 22:54:48.964203 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:54:48.964188    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36fd890f5a9056aed8ed4edd08d55d830176cd8dcd154e495781b9f543989969\": container with ID starting with 36fd890f5a9056aed8ed4edd08d55d830176cd8dcd154e495781b9f543989969 not found: ID does not exist" containerID="36fd890f5a9056aed8ed4edd08d55d830176cd8dcd154e495781b9f543989969"
Oct 30 22:54:48.964262 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:48.964205    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36fd890f5a9056aed8ed4edd08d55d830176cd8dcd154e495781b9f543989969"} err="failed to get container status \"36fd890f5a9056aed8ed4edd08d55d830176cd8dcd154e495781b9f543989969\": rpc error: code = NotFound desc = could not find container \"36fd890f5a9056aed8ed4edd08d55d830176cd8dcd154e495781b9f543989969\": container with ID starting with 36fd890f5a9056aed8ed4edd08d55d830176cd8dcd154e495781b9f543989969 not found: ID does not exist"
Oct 30 22:54:49.002487 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:49.002463    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:54:49.002487 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:49.002484    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:54:49.002631 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:49.002494    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:54:49.002631 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:49.002503    5896 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-cabundle-cert\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:54:49.002631 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:49.002511    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gb2zc\" (UniqueName: \"kubernetes.io/projected/ac0825f3-616f-4fbc-8fb0-3c879e753c5e-kube-api-access-gb2zc\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:54:49.169235 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:49.169197    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"]
Oct 30 22:54:49.176029 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:49.176003    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-promote-v2-predictor-66ffb95d77-svdzh"]
Oct 30 22:54:49.735784 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:49.735752    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" path="/var/lib/kubelet/pods/ac0825f3-616f-4fbc-8fb0-3c879e753c5e/volumes"
Oct 30 22:54:54.113263 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113227    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"]
Oct 30 22:54:54.113757 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113699    5896 cpu_manager.go:414] "containerMap: removing container" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kube-rbac-proxy"
Oct 30 22:54:54.113757 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113716    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kube-rbac-proxy"
Oct 30 22:54:54.113757 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113743    5896 cpu_manager.go:414] "containerMap: removing container" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="storage-initializer"
Oct 30 22:54:54.113757 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113750    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="storage-initializer"
Oct 30 22:54:54.113989 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113769    5896 cpu_manager.go:414] "containerMap: removing container" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="storage-initializer"
Oct 30 22:54:54.113989 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113776    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="storage-initializer"
Oct 30 22:54:54.113989 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113786    5896 cpu_manager.go:414] "containerMap: removing container" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kserve-container"
Oct 30 22:54:54.113989 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113793    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kserve-container"
Oct 30 22:54:54.113989 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113811    5896 cpu_manager.go:414] "containerMap: removing container" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kserve-container"
Oct 30 22:54:54.113989 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113818    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kserve-container"
Oct 30 22:54:54.113989 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113837    5896 cpu_manager.go:414] "containerMap: removing container" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kube-rbac-proxy"
Oct 30 22:54:54.113989 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113845    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kube-rbac-proxy"
Oct 30 22:54:54.113989 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113919    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kserve-container"
Oct 30 22:54:54.113989 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113933    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kube-rbac-proxy"
Oct 30 22:54:54.113989 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113942    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac0825f3-616f-4fbc-8fb0-3c879e753c5e" containerName="kube-rbac-proxy"
Oct 30 22:54:54.113989 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.113956    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="07cc7141-610c-4b63-a1a7-0df7af34437c" containerName="kserve-container"
Oct 30 22:54:54.118799 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.118780    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.142955 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.142933    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qscnt\" (UniqueName: \"kubernetes.io/projected/171e1f63-a52e-4c46-8d55-3c11f365f623-kube-api-access-qscnt\") pod \"isvc-canary-rollback-predictor-5597787b4-5mspg\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.143045 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.142966    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/171e1f63-a52e-4c46-8d55-3c11f365f623-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-5597787b4-5mspg\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.143045 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.142991    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/171e1f63-a52e-4c46-8d55-3c11f365f623-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-5597787b4-5mspg\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.143129 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.143087    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/171e1f63-a52e-4c46-8d55-3c11f365f623-cabundle-cert\") pod \"isvc-canary-rollback-predictor-5597787b4-5mspg\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.143172 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.143162    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/171e1f63-a52e-4c46-8d55-3c11f365f623-proxy-tls\") pod \"isvc-canary-rollback-predictor-5597787b4-5mspg\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.173512 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.173489    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"seaweedfs-s3-creds\""
Oct 30 22:54:54.223481 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.223455    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-canary-rollback-predictor-serving-cert\""
Oct 30 22:54:54.223584 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.223522    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"kube-root-ca.crt\""
Oct 30 22:54:54.223584 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.223524    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"openshift-service-ca.crt\""
Oct 30 22:54:54.223584 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.223556    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\""
Oct 30 22:54:54.223584 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.223570    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:54:54.223754 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.223590    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"default-dockercfg-s9cqw\""
Oct 30 22:54:54.224934 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.224918    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"]
Oct 30 22:54:54.243658 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.243626    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/171e1f63-a52e-4c46-8d55-3c11f365f623-cabundle-cert\") pod \"isvc-canary-rollback-predictor-5597787b4-5mspg\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.243775 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.243720    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/171e1f63-a52e-4c46-8d55-3c11f365f623-proxy-tls\") pod \"isvc-canary-rollback-predictor-5597787b4-5mspg\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.243775 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.243762    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qscnt\" (UniqueName: \"kubernetes.io/projected/171e1f63-a52e-4c46-8d55-3c11f365f623-kube-api-access-qscnt\") pod \"isvc-canary-rollback-predictor-5597787b4-5mspg\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.243893 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.243804    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/171e1f63-a52e-4c46-8d55-3c11f365f623-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-5597787b4-5mspg\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.243893 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.243842    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/171e1f63-a52e-4c46-8d55-3c11f365f623-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-5597787b4-5mspg\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.244217 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.244198    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/171e1f63-a52e-4c46-8d55-3c11f365f623-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-5597787b4-5mspg\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.244299 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.244244    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/171e1f63-a52e-4c46-8d55-3c11f365f623-cabundle-cert\") pod \"isvc-canary-rollback-predictor-5597787b4-5mspg\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.244420 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.244400    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/171e1f63-a52e-4c46-8d55-3c11f365f623-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-5597787b4-5mspg\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.246292 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.246273    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/171e1f63-a52e-4c46-8d55-3c11f365f623-proxy-tls\") pod \"isvc-canary-rollback-predictor-5597787b4-5mspg\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.307401 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.307381    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qscnt\" (UniqueName: \"kubernetes.io/projected/171e1f63-a52e-4c46-8d55-3c11f365f623-kube-api-access-qscnt\") pod \"isvc-canary-rollback-predictor-5597787b4-5mspg\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.435246 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.435174    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:54.858442 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.858419    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"]
Oct 30 22:54:54.861073 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:54:54.861031    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod171e1f63_a52e_4c46_8d55_3c11f365f623.slice/crio-3c11f05a726242a5f6f7ad2c7206b0b6fdf4920cd69f9997566388fbd908f96b WatchSource:0}: Error finding container 3c11f05a726242a5f6f7ad2c7206b0b6fdf4920cd69f9997566388fbd908f96b: Status 404 returned error can't find the container with id 3c11f05a726242a5f6f7ad2c7206b0b6fdf4920cd69f9997566388fbd908f96b
Oct 30 22:54:54.963155 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.963125    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" event={"ID":"171e1f63-a52e-4c46-8d55-3c11f365f623","Type":"ContainerStarted","Data":"00e3aea37e69f8b6e4e161e7bf147c333ed038a4ba2f1c02fc5d02e88fab2a86"}
Oct 30 22:54:54.963269 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:54.963179    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" event={"ID":"171e1f63-a52e-4c46-8d55-3c11f365f623","Type":"ContainerStarted","Data":"3c11f05a726242a5f6f7ad2c7206b0b6fdf4920cd69f9997566388fbd908f96b"}
Oct 30 22:54:58.982130 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:58.982051    5896 generic.go:356] "Generic (PLEG): container finished" podID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerID="00e3aea37e69f8b6e4e161e7bf147c333ed038a4ba2f1c02fc5d02e88fab2a86" exitCode=0
Oct 30 22:54:58.982498 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:58.982125    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" event={"ID":"171e1f63-a52e-4c46-8d55-3c11f365f623","Type":"ContainerDied","Data":"00e3aea37e69f8b6e4e161e7bf147c333ed038a4ba2f1c02fc5d02e88fab2a86"}
Oct 30 22:54:59.988822 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:59.988784    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" event={"ID":"171e1f63-a52e-4c46-8d55-3c11f365f623","Type":"ContainerStarted","Data":"5d5a8ff14c7f316ac1b7fd39d7118f4cf8b0ab43a005905800614bfebe4a4d7d"}
Oct 30 22:54:59.989292 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:59.988860    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" event={"ID":"171e1f63-a52e-4c46-8d55-3c11f365f623","Type":"ContainerStarted","Data":"67a3b587ff3dd562d0f8ee5895e20b46194c43ebc99874a759472e8b2f115085"}
Oct 30 22:54:59.989292 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:59.989133    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:59.989292 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:59.989187    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:54:59.990342 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:54:59.990318    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 30 22:55:00.161886 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:55:00.161838    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" podStartSLOduration=2.355395277 podStartE2EDuration="6.161824519s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.806429242s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:54:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:55:00.160939857 +0000 UTC m=+2752.572405824" watchObservedRunningTime="2026-10-30 22:55:00.161824519 +0000 UTC m=+2752.573290486"
Oct 30 22:55:00.993635 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:55:00.993589    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 30 22:55:02.626437 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:55:02.626403    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:55:05.998609 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:55:05.998578    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:55:05.999098 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:55:05.999073    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 30 22:55:15.999222 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:55:15.999180    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 30 22:55:25.999176 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:55:25.999138    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 30 22:55:32.627491 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:55:32.627446    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:55:35.999331 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:55:35.999294    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 30 22:55:45.999982 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:55:45.999893    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 30 22:55:55.999955 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:55:55.999914    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 30 22:56:02.229204 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.229164    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"]
Oct 30 22:56:02.234730 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.234705    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:02.259802 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.259767    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pl25\" (UniqueName: \"kubernetes.io/projected/d4258393-5668-45c3-aca9-e3517715ec6b-kube-api-access-9pl25\") pod \"isvc-sklearn-s3-predictor-6768889f44-rrn2f\" (UID: \"d4258393-5668-45c3-aca9-e3517715ec6b\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:02.259802 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.259806    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d4258393-5668-45c3-aca9-e3517715ec6b-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-s3-predictor-6768889f44-rrn2f\" (UID: \"d4258393-5668-45c3-aca9-e3517715ec6b\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:02.259957 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.259826    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d4258393-5668-45c3-aca9-e3517715ec6b-kserve-provision-location\") pod \"isvc-sklearn-s3-predictor-6768889f44-rrn2f\" (UID: \"d4258393-5668-45c3-aca9-e3517715ec6b\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:02.260007 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.259988    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d4258393-5668-45c3-aca9-e3517715ec6b-proxy-tls\") pod \"isvc-sklearn-s3-predictor-6768889f44-rrn2f\" (UID: \"d4258393-5668-45c3-aca9-e3517715ec6b\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:02.339324 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.339300    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"kube-root-ca.crt\""
Oct 30 22:56:02.339466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.339441    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-sklearn-s3-kube-rbac-proxy-sar-config\""
Oct 30 22:56:02.339466 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.339327    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"default-dockercfg-nnb5w\""
Oct 30 22:56:02.339656 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.339383    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"openshift-service-ca.crt\""
Oct 30 22:56:02.339656 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.339315    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"storage-config\""
Oct 30 22:56:02.339656 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.339487    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-sklearn-s3-predictor-serving-cert\""
Oct 30 22:56:02.341003 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.340988    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"]
Oct 30 22:56:02.360793 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.360770    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d4258393-5668-45c3-aca9-e3517715ec6b-proxy-tls\") pod \"isvc-sklearn-s3-predictor-6768889f44-rrn2f\" (UID: \"d4258393-5668-45c3-aca9-e3517715ec6b\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:02.360899 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.360818    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pl25\" (UniqueName: \"kubernetes.io/projected/d4258393-5668-45c3-aca9-e3517715ec6b-kube-api-access-9pl25\") pod \"isvc-sklearn-s3-predictor-6768889f44-rrn2f\" (UID: \"d4258393-5668-45c3-aca9-e3517715ec6b\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:02.360899 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.360842    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d4258393-5668-45c3-aca9-e3517715ec6b-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-s3-predictor-6768889f44-rrn2f\" (UID: \"d4258393-5668-45c3-aca9-e3517715ec6b\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:02.360899 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.360859    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d4258393-5668-45c3-aca9-e3517715ec6b-kserve-provision-location\") pod \"isvc-sklearn-s3-predictor-6768889f44-rrn2f\" (UID: \"d4258393-5668-45c3-aca9-e3517715ec6b\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:02.361200 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.361183    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d4258393-5668-45c3-aca9-e3517715ec6b-kserve-provision-location\") pod \"isvc-sklearn-s3-predictor-6768889f44-rrn2f\" (UID: \"d4258393-5668-45c3-aca9-e3517715ec6b\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:02.361376 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.361360    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d4258393-5668-45c3-aca9-e3517715ec6b-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-s3-predictor-6768889f44-rrn2f\" (UID: \"d4258393-5668-45c3-aca9-e3517715ec6b\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:02.363540 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.363521    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d4258393-5668-45c3-aca9-e3517715ec6b-proxy-tls\") pod \"isvc-sklearn-s3-predictor-6768889f44-rrn2f\" (UID: \"d4258393-5668-45c3-aca9-e3517715ec6b\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:02.425118 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.425086    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pl25\" (UniqueName: \"kubernetes.io/projected/d4258393-5668-45c3-aca9-e3517715ec6b-kube-api-access-9pl25\") pod \"isvc-sklearn-s3-predictor-6768889f44-rrn2f\" (UID: \"d4258393-5668-45c3-aca9-e3517715ec6b\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:02.552196 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.552120    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:02.632710 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:56:02.632675    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:56:02.978293 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:02.978268    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"]
Oct 30 22:56:02.980305 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:56:02.980278    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4258393_5668_45c3_aca9_e3517715ec6b.slice/crio-b5e69f04a9422ce2bc3f714a87dd1963f5f61e484345836a696b48cb020b94cc WatchSource:0}: Error finding container b5e69f04a9422ce2bc3f714a87dd1963f5f61e484345836a696b48cb020b94cc: Status 404 returned error can't find the container with id b5e69f04a9422ce2bc3f714a87dd1963f5f61e484345836a696b48cb020b94cc
Oct 30 22:56:03.246180 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:03.246088    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" event={"ID":"d4258393-5668-45c3-aca9-e3517715ec6b","Type":"ContainerStarted","Data":"516af6937305b74ccaf6ad3c1825ff4b40bbda277cc73872f4d3414265c0e5df"}
Oct 30 22:56:03.246180 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:03.246154    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" event={"ID":"d4258393-5668-45c3-aca9-e3517715ec6b","Type":"ContainerStarted","Data":"b5e69f04a9422ce2bc3f714a87dd1963f5f61e484345836a696b48cb020b94cc"}
Oct 30 22:56:04.252436 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:04.252405    5896 generic.go:356] "Generic (PLEG): container finished" podID="d4258393-5668-45c3-aca9-e3517715ec6b" containerID="516af6937305b74ccaf6ad3c1825ff4b40bbda277cc73872f4d3414265c0e5df" exitCode=0
Oct 30 22:56:04.252794 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:04.252475    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" event={"ID":"d4258393-5668-45c3-aca9-e3517715ec6b","Type":"ContainerDied","Data":"516af6937305b74ccaf6ad3c1825ff4b40bbda277cc73872f4d3414265c0e5df"}
Oct 30 22:56:05.258909 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:05.258876    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" event={"ID":"d4258393-5668-45c3-aca9-e3517715ec6b","Type":"ContainerStarted","Data":"ca4ac9be996a274b0fe7019863fb68e504101963237ace31cf1c5478cc9bffd2"}
Oct 30 22:56:05.259287 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:05.258934    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" event={"ID":"d4258393-5668-45c3-aca9-e3517715ec6b","Type":"ContainerStarted","Data":"145b93ee16ec6d67f773eb30299d8d15ee75dea61f0f349924f7b0efff748b51"}
Oct 30 22:56:05.259287 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:05.259034    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:05.381993 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:05.381940    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" podStartSLOduration=2.32278749 podStartE2EDuration="3.381923561s" totalImagesPullingTime="0s" totalInitContainerRuntime="1.059136071s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:56:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:56:05.381869279 +0000 UTC m=+2817.793335246" watchObservedRunningTime="2026-10-30 22:56:05.381923561 +0000 UTC m=+2817.793389528"
Oct 30 22:56:05.999684 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:05.999626    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 30 22:56:06.263440 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:06.263366    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:06.264419 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:06.264393    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:56:07.268236 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:07.268194    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:56:12.274259 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:12.274228    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:56:12.274768 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:12.274685    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:56:16.000316 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:16.000287    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:56:22.275691 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:22.275634    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:56:26.757217 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:26.757184    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"]
Oct 30 22:56:26.757654 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:26.757450    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kserve-container" containerID="cri-o://67a3b587ff3dd562d0f8ee5895e20b46194c43ebc99874a759472e8b2f115085" gracePeriod=30
Oct 30 22:56:26.757654 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:26.757501    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kube-rbac-proxy" containerID="cri-o://5d5a8ff14c7f316ac1b7fd39d7118f4cf8b0ab43a005905800614bfebe4a4d7d" gracePeriod=30
Oct 30 22:56:27.358785 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:27.358755    5896 generic.go:356] "Generic (PLEG): container finished" podID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerID="5d5a8ff14c7f316ac1b7fd39d7118f4cf8b0ab43a005905800614bfebe4a4d7d" exitCode=0
Oct 30 22:56:27.358961 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:27.358838    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" event={"ID":"171e1f63-a52e-4c46-8d55-3c11f365f623","Type":"ContainerDied","Data":"5d5a8ff14c7f316ac1b7fd39d7118f4cf8b0ab43a005905800614bfebe4a4d7d"}
Oct 30 22:56:29.474569 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.474536    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"]
Oct 30 22:56:29.478145 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.478130    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:29.582682 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.582625    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-cabundle-cert\") pod \"isvc-canary-rollback-predictor-5597787b4-pc4ht\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:29.582682 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.582682    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-5597787b4-pc4ht\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:29.582932 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.582799    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbqpf\" (UniqueName: \"kubernetes.io/projected/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kube-api-access-zbqpf\") pod \"isvc-canary-rollback-predictor-5597787b4-pc4ht\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:29.582932 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.582829    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-5597787b4-pc4ht\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:29.583038 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.582942    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-proxy-tls\") pod \"isvc-canary-rollback-predictor-5597787b4-pc4ht\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:29.583747 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.583722    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"]
Oct 30 22:56:29.683526 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.683493    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-proxy-tls\") pod \"isvc-canary-rollback-predictor-5597787b4-pc4ht\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:29.683710 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.683546    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-cabundle-cert\") pod \"isvc-canary-rollback-predictor-5597787b4-pc4ht\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:29.683710 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.683574    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-5597787b4-pc4ht\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:29.683710 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.683615    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbqpf\" (UniqueName: \"kubernetes.io/projected/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kube-api-access-zbqpf\") pod \"isvc-canary-rollback-predictor-5597787b4-pc4ht\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:29.683710 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.683667    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-5597787b4-pc4ht\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:29.684109 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.684086    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-5597787b4-pc4ht\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:29.684249 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.684230    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-cabundle-cert\") pod \"isvc-canary-rollback-predictor-5597787b4-pc4ht\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:29.684335 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.684320    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-5597787b4-pc4ht\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:29.686184 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.686147    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-proxy-tls\") pod \"isvc-canary-rollback-predictor-5597787b4-pc4ht\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:29.747057 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.747005    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbqpf\" (UniqueName: \"kubernetes.io/projected/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kube-api-access-zbqpf\") pod \"isvc-canary-rollback-predictor-5597787b4-pc4ht\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") " pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:29.794673 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:29.794631    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:30.217471 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:30.217447    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"]
Oct 30 22:56:30.220250 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:56:30.220223    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b594ed4_bdc1_4d5b_a2b4_2d6576cced36.slice/crio-db767f56a8d1a4da1c4ffe64dc0c2e7b4cfcddd157700edbd1586ac2cb6b8ca2 WatchSource:0}: Error finding container db767f56a8d1a4da1c4ffe64dc0c2e7b4cfcddd157700edbd1586ac2cb6b8ca2: Status 404 returned error can't find the container with id db767f56a8d1a4da1c4ffe64dc0c2e7b4cfcddd157700edbd1586ac2cb6b8ca2
Oct 30 22:56:30.377791 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:30.377753    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" event={"ID":"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36","Type":"ContainerStarted","Data":"241000e857eef725b6a524452959d2bdffc87ff0c4b15e2c0bf342cba7e58790"}
Oct 30 22:56:30.377954 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:30.377833    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" event={"ID":"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36","Type":"ContainerStarted","Data":"db767f56a8d1a4da1c4ffe64dc0c2e7b4cfcddd157700edbd1586ac2cb6b8ca2"}
Oct 30 22:56:30.993946 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:30.993906    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Oct 30 22:56:31.385567 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.385530    5896 generic.go:356] "Generic (PLEG): container finished" podID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerID="67a3b587ff3dd562d0f8ee5895e20b46194c43ebc99874a759472e8b2f115085" exitCode=0
Oct 30 22:56:31.385706 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.385590    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" event={"ID":"171e1f63-a52e-4c46-8d55-3c11f365f623","Type":"ContainerDied","Data":"67a3b587ff3dd562d0f8ee5895e20b46194c43ebc99874a759472e8b2f115085"}
Oct 30 22:56:31.385706 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.385662    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg" event={"ID":"171e1f63-a52e-4c46-8d55-3c11f365f623","Type":"ContainerDied","Data":"3c11f05a726242a5f6f7ad2c7206b0b6fdf4920cd69f9997566388fbd908f96b"}
Oct 30 22:56:31.385706 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.385686    5896 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c11f05a726242a5f6f7ad2c7206b0b6fdf4920cd69f9997566388fbd908f96b"
Oct 30 22:56:31.395186 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.395169    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:56:31.498240 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.498215    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/171e1f63-a52e-4c46-8d55-3c11f365f623-proxy-tls\" (UniqueName: \"kubernetes.io/secret/171e1f63-a52e-4c46-8d55-3c11f365f623-proxy-tls\") pod \"171e1f63-a52e-4c46-8d55-3c11f365f623\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") "
Oct 30 22:56:31.498355 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.498250    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/171e1f63-a52e-4c46-8d55-3c11f365f623-kube-api-access-qscnt\" (UniqueName: \"kubernetes.io/projected/171e1f63-a52e-4c46-8d55-3c11f365f623-kube-api-access-qscnt\") pod \"171e1f63-a52e-4c46-8d55-3c11f365f623\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") "
Oct 30 22:56:31.498355 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.498274    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/171e1f63-a52e-4c46-8d55-3c11f365f623-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/171e1f63-a52e-4c46-8d55-3c11f365f623-kube-rbac-proxy-sar-config\") pod \"171e1f63-a52e-4c46-8d55-3c11f365f623\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") "
Oct 30 22:56:31.498355 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.498291    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/171e1f63-a52e-4c46-8d55-3c11f365f623-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/171e1f63-a52e-4c46-8d55-3c11f365f623-cabundle-cert\") pod \"171e1f63-a52e-4c46-8d55-3c11f365f623\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") "
Oct 30 22:56:31.498355 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.498326    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/171e1f63-a52e-4c46-8d55-3c11f365f623-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/171e1f63-a52e-4c46-8d55-3c11f365f623-kserve-provision-location\") pod \"171e1f63-a52e-4c46-8d55-3c11f365f623\" (UID: \"171e1f63-a52e-4c46-8d55-3c11f365f623\") "
Oct 30 22:56:31.498680 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.498626    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/171e1f63-a52e-4c46-8d55-3c11f365f623-kube-rbac-proxy-sar-config" pod "171e1f63-a52e-4c46-8d55-3c11f365f623" (UID: "171e1f63-a52e-4c46-8d55-3c11f365f623"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:56:31.498855 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.498827    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/171e1f63-a52e-4c46-8d55-3c11f365f623-cabundle-cert" pod "171e1f63-a52e-4c46-8d55-3c11f365f623" (UID: "171e1f63-a52e-4c46-8d55-3c11f365f623"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:56:31.498962 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.498856    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/171e1f63-a52e-4c46-8d55-3c11f365f623-kserve-provision-location" pod "171e1f63-a52e-4c46-8d55-3c11f365f623" (UID: "171e1f63-a52e-4c46-8d55-3c11f365f623"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:56:31.500552 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.500536    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/171e1f63-a52e-4c46-8d55-3c11f365f623-kube-api-access-qscnt" pod "171e1f63-a52e-4c46-8d55-3c11f365f623" (UID: "171e1f63-a52e-4c46-8d55-3c11f365f623"). InnerVolumeSpecName "kube-api-access-qscnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:56:31.500768 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.500748    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/171e1f63-a52e-4c46-8d55-3c11f365f623-proxy-tls" pod "171e1f63-a52e-4c46-8d55-3c11f365f623" (UID: "171e1f63-a52e-4c46-8d55-3c11f365f623"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:56:31.598987 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.598930    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/171e1f63-a52e-4c46-8d55-3c11f365f623-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:56:31.598987 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.598954    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qscnt\" (UniqueName: \"kubernetes.io/projected/171e1f63-a52e-4c46-8d55-3c11f365f623-kube-api-access-qscnt\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:56:31.598987 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.598964    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/171e1f63-a52e-4c46-8d55-3c11f365f623-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:56:31.598987 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.598973    5896 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/171e1f63-a52e-4c46-8d55-3c11f365f623-cabundle-cert\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:56:31.598987 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:31.598983    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/171e1f63-a52e-4c46-8d55-3c11f365f623-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:56:32.275181 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:32.275143    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:56:32.390676 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:32.390628    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"
Oct 30 22:56:32.582748 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:32.582664    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"]
Oct 30 22:56:32.628658 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:56:32.628608    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:56:32.633657 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:32.633618    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-5mspg"]
Oct 30 22:56:33.735654 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:33.735593    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" path="/var/lib/kubelet/pods/171e1f63-a52e-4c46-8d55-3c11f365f623/volumes"
Oct 30 22:56:34.401354 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:34.401326    5896 generic.go:356] "Generic (PLEG): container finished" podID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerID="241000e857eef725b6a524452959d2bdffc87ff0c4b15e2c0bf342cba7e58790" exitCode=0
Oct 30 22:56:34.401488 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:34.401404    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" event={"ID":"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36","Type":"ContainerDied","Data":"241000e857eef725b6a524452959d2bdffc87ff0c4b15e2c0bf342cba7e58790"}
Oct 30 22:56:35.408509 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:35.408473    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" event={"ID":"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36","Type":"ContainerStarted","Data":"bfd4a3573dbe7b64ed89328b42bd5f4d77069514436f715c041ee52ccbe5e5ad"}
Oct 30 22:56:35.408913 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:35.408541    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" event={"ID":"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36","Type":"ContainerStarted","Data":"68e353f8bdc824dbbee9d0f1b9ffd210b00efdc78fb468af21637b84b4d99643"}
Oct 30 22:56:35.408913 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:35.408753    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:35.532293 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:35.532235    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" podStartSLOduration=2.561831898 podStartE2EDuration="6.532222492s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.970390594s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:56:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:56:35.531989907 +0000 UTC m=+2847.943455875" watchObservedRunningTime="2026-10-30 22:56:35.532222492 +0000 UTC m=+2847.943688455"
Oct 30 22:56:36.413205 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:36.413168    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:36.414282 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:36.414258    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 30 22:56:37.417896 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:37.417859    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 30 22:56:42.275085 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:42.275045    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:56:42.422945 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:42.422919    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:56:42.423390 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:42.423361    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 30 22:56:52.274696 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:52.274656    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:56:52.424188 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:56:52.424151    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 30 22:57:02.275012 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:02.274972    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:57:02.423358 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:02.423323    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 30 22:57:02.627567 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:57:02.627480    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:57:12.274804 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:12.274757    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:57:12.423917 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:12.423871    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 30 22:57:17.736504 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:17.736417    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:57:22.423785 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:22.423746    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 30 22:57:22.928215 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:22.928181    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"]
Oct 30 22:57:22.928444 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:22.928406    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kserve-container" containerID="cri-o://145b93ee16ec6d67f773eb30299d8d15ee75dea61f0f349924f7b0efff748b51" gracePeriod=30
Oct 30 22:57:22.928505 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:22.928441    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kube-rbac-proxy" containerID="cri-o://ca4ac9be996a274b0fe7019863fb68e504101963237ace31cf1c5478cc9bffd2" gracePeriod=30
Oct 30 22:57:23.626447 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:23.626416    5896 generic.go:356] "Generic (PLEG): container finished" podID="d4258393-5668-45c3-aca9-e3517715ec6b" containerID="ca4ac9be996a274b0fe7019863fb68e504101963237ace31cf1c5478cc9bffd2" exitCode=0
Oct 30 22:57:23.626840 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:23.626461    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" event={"ID":"d4258393-5668-45c3-aca9-e3517715ec6b","Type":"ContainerDied","Data":"ca4ac9be996a274b0fe7019863fb68e504101963237ace31cf1c5478cc9bffd2"}
Oct 30 22:57:27.269161 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.269110    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Oct 30 22:57:27.471329 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.471303    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:57:27.550576 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.550513    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4258393-5668-45c3-aca9-e3517715ec6b-proxy-tls\" (UniqueName: \"kubernetes.io/secret/d4258393-5668-45c3-aca9-e3517715ec6b-proxy-tls\") pod \"d4258393-5668-45c3-aca9-e3517715ec6b\" (UID: \"d4258393-5668-45c3-aca9-e3517715ec6b\") "
Oct 30 22:57:27.550725 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.550598    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4258393-5668-45c3-aca9-e3517715ec6b-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d4258393-5668-45c3-aca9-e3517715ec6b-kserve-provision-location\") pod \"d4258393-5668-45c3-aca9-e3517715ec6b\" (UID: \"d4258393-5668-45c3-aca9-e3517715ec6b\") "
Oct 30 22:57:27.550725 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.550711    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4258393-5668-45c3-aca9-e3517715ec6b-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d4258393-5668-45c3-aca9-e3517715ec6b-kube-rbac-proxy-sar-config\") pod \"d4258393-5668-45c3-aca9-e3517715ec6b\" (UID: \"d4258393-5668-45c3-aca9-e3517715ec6b\") "
Oct 30 22:57:27.550854 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.550746    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4258393-5668-45c3-aca9-e3517715ec6b-kube-api-access-9pl25\" (UniqueName: \"kubernetes.io/projected/d4258393-5668-45c3-aca9-e3517715ec6b-kube-api-access-9pl25\") pod \"d4258393-5668-45c3-aca9-e3517715ec6b\" (UID: \"d4258393-5668-45c3-aca9-e3517715ec6b\") "
Oct 30 22:57:27.550963 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.550934    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4258393-5668-45c3-aca9-e3517715ec6b-kserve-provision-location" pod "d4258393-5668-45c3-aca9-e3517715ec6b" (UID: "d4258393-5668-45c3-aca9-e3517715ec6b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:57:27.551041 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.551022    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4258393-5668-45c3-aca9-e3517715ec6b-kube-rbac-proxy-sar-config" pod "d4258393-5668-45c3-aca9-e3517715ec6b" (UID: "d4258393-5668-45c3-aca9-e3517715ec6b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:57:27.552733 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.552715    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4258393-5668-45c3-aca9-e3517715ec6b-proxy-tls" pod "d4258393-5668-45c3-aca9-e3517715ec6b" (UID: "d4258393-5668-45c3-aca9-e3517715ec6b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:57:27.552920 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.552895    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4258393-5668-45c3-aca9-e3517715ec6b-kube-api-access-9pl25" pod "d4258393-5668-45c3-aca9-e3517715ec6b" (UID: "d4258393-5668-45c3-aca9-e3517715ec6b"). InnerVolumeSpecName "kube-api-access-9pl25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:57:27.651159 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.651131    5896 generic.go:356] "Generic (PLEG): container finished" podID="d4258393-5668-45c3-aca9-e3517715ec6b" containerID="145b93ee16ec6d67f773eb30299d8d15ee75dea61f0f349924f7b0efff748b51" exitCode=0
Oct 30 22:57:27.651293 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.651188    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"
Oct 30 22:57:27.651293 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.651244    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" event={"ID":"d4258393-5668-45c3-aca9-e3517715ec6b","Type":"ContainerDied","Data":"145b93ee16ec6d67f773eb30299d8d15ee75dea61f0f349924f7b0efff748b51"}
Oct 30 22:57:27.651374 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.651318    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f" event={"ID":"d4258393-5668-45c3-aca9-e3517715ec6b","Type":"ContainerDied","Data":"b5e69f04a9422ce2bc3f714a87dd1963f5f61e484345836a696b48cb020b94cc"}
Oct 30 22:57:27.651374 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.651345    5896 scope.go:122] "RemoveContainer" containerID="ca4ac9be996a274b0fe7019863fb68e504101963237ace31cf1c5478cc9bffd2"
Oct 30 22:57:27.651545 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.651523    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d4258393-5668-45c3-aca9-e3517715ec6b-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:57:27.651710 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.651686    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d4258393-5668-45c3-aca9-e3517715ec6b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:57:27.651839 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.651714    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pl25\" (UniqueName: \"kubernetes.io/projected/d4258393-5668-45c3-aca9-e3517715ec6b-kube-api-access-9pl25\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:57:27.651839 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.651731    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d4258393-5668-45c3-aca9-e3517715ec6b-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:57:27.663740 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.663721    5896 scope.go:122] "RemoveContainer" containerID="145b93ee16ec6d67f773eb30299d8d15ee75dea61f0f349924f7b0efff748b51"
Oct 30 22:57:27.672333 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.672314    5896 scope.go:122] "RemoveContainer" containerID="516af6937305b74ccaf6ad3c1825ff4b40bbda277cc73872f4d3414265c0e5df"
Oct 30 22:57:27.679583 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.679555    5896 scope.go:122] "RemoveContainer" containerID="ca4ac9be996a274b0fe7019863fb68e504101963237ace31cf1c5478cc9bffd2"
Oct 30 22:57:27.679865 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:57:27.679848    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca4ac9be996a274b0fe7019863fb68e504101963237ace31cf1c5478cc9bffd2\": container with ID starting with ca4ac9be996a274b0fe7019863fb68e504101963237ace31cf1c5478cc9bffd2 not found: ID does not exist" containerID="ca4ac9be996a274b0fe7019863fb68e504101963237ace31cf1c5478cc9bffd2"
Oct 30 22:57:27.679931 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.679871    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca4ac9be996a274b0fe7019863fb68e504101963237ace31cf1c5478cc9bffd2"} err="failed to get container status \"ca4ac9be996a274b0fe7019863fb68e504101963237ace31cf1c5478cc9bffd2\": rpc error: code = NotFound desc = could not find container \"ca4ac9be996a274b0fe7019863fb68e504101963237ace31cf1c5478cc9bffd2\": container with ID starting with ca4ac9be996a274b0fe7019863fb68e504101963237ace31cf1c5478cc9bffd2 not found: ID does not exist"
Oct 30 22:57:27.679931 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.679895    5896 scope.go:122] "RemoveContainer" containerID="145b93ee16ec6d67f773eb30299d8d15ee75dea61f0f349924f7b0efff748b51"
Oct 30 22:57:27.680130 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:57:27.680109    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"145b93ee16ec6d67f773eb30299d8d15ee75dea61f0f349924f7b0efff748b51\": container with ID starting with 145b93ee16ec6d67f773eb30299d8d15ee75dea61f0f349924f7b0efff748b51 not found: ID does not exist" containerID="145b93ee16ec6d67f773eb30299d8d15ee75dea61f0f349924f7b0efff748b51"
Oct 30 22:57:27.680171 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.680142    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"145b93ee16ec6d67f773eb30299d8d15ee75dea61f0f349924f7b0efff748b51"} err="failed to get container status \"145b93ee16ec6d67f773eb30299d8d15ee75dea61f0f349924f7b0efff748b51\": rpc error: code = NotFound desc = could not find container \"145b93ee16ec6d67f773eb30299d8d15ee75dea61f0f349924f7b0efff748b51\": container with ID starting with 145b93ee16ec6d67f773eb30299d8d15ee75dea61f0f349924f7b0efff748b51 not found: ID does not exist"
Oct 30 22:57:27.680171 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.680158    5896 scope.go:122] "RemoveContainer" containerID="516af6937305b74ccaf6ad3c1825ff4b40bbda277cc73872f4d3414265c0e5df"
Oct 30 22:57:27.680381 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:57:27.680366    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"516af6937305b74ccaf6ad3c1825ff4b40bbda277cc73872f4d3414265c0e5df\": container with ID starting with 516af6937305b74ccaf6ad3c1825ff4b40bbda277cc73872f4d3414265c0e5df not found: ID does not exist" containerID="516af6937305b74ccaf6ad3c1825ff4b40bbda277cc73872f4d3414265c0e5df"
Oct 30 22:57:27.680427 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.680385    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"516af6937305b74ccaf6ad3c1825ff4b40bbda277cc73872f4d3414265c0e5df"} err="failed to get container status \"516af6937305b74ccaf6ad3c1825ff4b40bbda277cc73872f4d3414265c0e5df\": rpc error: code = NotFound desc = could not find container \"516af6937305b74ccaf6ad3c1825ff4b40bbda277cc73872f4d3414265c0e5df\": container with ID starting with 516af6937305b74ccaf6ad3c1825ff4b40bbda277cc73872f4d3414265c0e5df not found: ID does not exist"
Oct 30 22:57:27.820459 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.820397    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"]
Oct 30 22:57:27.826209 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:27.826186    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-predictor-6768889f44-rrn2f"]
Oct 30 22:57:29.735423 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:29.735386    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" path="/var/lib/kubelet/pods/d4258393-5668-45c3-aca9-e3517715ec6b/volumes"
Oct 30 22:57:32.423353 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:32.423312    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 30 22:57:32.628002 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:57:32.627959    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:57:42.423529 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:42.423484    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 30 22:57:52.423803 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:52.423769    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:57:59.000342 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:59.000315    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"]
Oct 30 22:57:59.000802 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:59.000534    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kserve-container" containerID="cri-o://68e353f8bdc824dbbee9d0f1b9ffd210b00efdc78fb468af21637b84b4d99643" gracePeriod=30
Oct 30 22:57:59.000802 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:59.000568    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kube-rbac-proxy" containerID="cri-o://bfd4a3573dbe7b64ed89328b42bd5f4d77069514436f715c041ee52ccbe5e5ad" gracePeriod=30
Oct 30 22:57:59.794053 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:59.794022    5896 generic.go:356] "Generic (PLEG): container finished" podID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerID="bfd4a3573dbe7b64ed89328b42bd5f4d77069514436f715c041ee52ccbe5e5ad" exitCode=0
Oct 30 22:57:59.794218 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:57:59.794066    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" event={"ID":"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36","Type":"ContainerDied","Data":"bfd4a3573dbe7b64ed89328b42bd5f4d77069514436f715c041ee52ccbe5e5ad"}
Oct 30 22:58:02.417956 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:02.417915    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.35:8643/healthz\": dial tcp 10.134.0.35:8643: connect: connection refused"
Oct 30 22:58:02.423318 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:02.423283    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 30 22:58:02.626329 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:58:02.626295    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:58:03.453324 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.453302    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:58:03.522583 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.522533    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kserve-provision-location\") pod \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") "
Oct 30 22:58:03.522709 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.522597    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kube-rbac-proxy-sar-config\") pod \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") "
Oct 30 22:58:03.522709 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.522623    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-cabundle-cert\") pod \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") "
Oct 30 22:58:03.522709 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.522666    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-proxy-tls\" (UniqueName: \"kubernetes.io/secret/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-proxy-tls\") pod \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") "
Oct 30 22:58:03.522846 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.522754    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kube-api-access-zbqpf\" (UniqueName: \"kubernetes.io/projected/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kube-api-access-zbqpf\") pod \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\" (UID: \"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36\") "
Oct 30 22:58:03.522948 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.522886    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kserve-provision-location" pod "9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" (UID: "9b594ed4-bdc1-4d5b-a2b4-2d6576cced36"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:58:03.523007 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.522979    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kube-rbac-proxy-sar-config" pod "9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" (UID: "9b594ed4-bdc1-4d5b-a2b4-2d6576cced36"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:58:03.523052 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.523036    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:58:03.523093 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.523048    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:58:03.523172 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.523151    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-cabundle-cert" pod "9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" (UID: "9b594ed4-bdc1-4d5b-a2b4-2d6576cced36"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:58:03.525087 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.525066    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-proxy-tls" pod "9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" (UID: "9b594ed4-bdc1-4d5b-a2b4-2d6576cced36"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:58:03.525087 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.525080    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kube-api-access-zbqpf" pod "9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" (UID: "9b594ed4-bdc1-4d5b-a2b4-2d6576cced36"). InnerVolumeSpecName "kube-api-access-zbqpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:58:03.624254 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.624232    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:58:03.624254 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.624251    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbqpf\" (UniqueName: \"kubernetes.io/projected/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-kube-api-access-zbqpf\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:58:03.624386 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.624260    5896 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36-cabundle-cert\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:58:03.817155 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.817079    5896 generic.go:356] "Generic (PLEG): container finished" podID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerID="68e353f8bdc824dbbee9d0f1b9ffd210b00efdc78fb468af21637b84b4d99643" exitCode=0
Oct 30 22:58:03.817278 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.817149    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"
Oct 30 22:58:03.817278 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.817152    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" event={"ID":"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36","Type":"ContainerDied","Data":"68e353f8bdc824dbbee9d0f1b9ffd210b00efdc78fb468af21637b84b4d99643"}
Oct 30 22:58:03.817278 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.817238    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht" event={"ID":"9b594ed4-bdc1-4d5b-a2b4-2d6576cced36","Type":"ContainerDied","Data":"db767f56a8d1a4da1c4ffe64dc0c2e7b4cfcddd157700edbd1586ac2cb6b8ca2"}
Oct 30 22:58:03.817278 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.817266    5896 scope.go:122] "RemoveContainer" containerID="bfd4a3573dbe7b64ed89328b42bd5f4d77069514436f715c041ee52ccbe5e5ad"
Oct 30 22:58:03.825497 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.825478    5896 scope.go:122] "RemoveContainer" containerID="68e353f8bdc824dbbee9d0f1b9ffd210b00efdc78fb468af21637b84b4d99643"
Oct 30 22:58:03.832752 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.832733    5896 scope.go:122] "RemoveContainer" containerID="241000e857eef725b6a524452959d2bdffc87ff0c4b15e2c0bf342cba7e58790"
Oct 30 22:58:03.839733 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.839715    5896 scope.go:122] "RemoveContainer" containerID="bfd4a3573dbe7b64ed89328b42bd5f4d77069514436f715c041ee52ccbe5e5ad"
Oct 30 22:58:03.840003 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:58:03.839977    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfd4a3573dbe7b64ed89328b42bd5f4d77069514436f715c041ee52ccbe5e5ad\": container with ID starting with bfd4a3573dbe7b64ed89328b42bd5f4d77069514436f715c041ee52ccbe5e5ad not found: ID does not exist" containerID="bfd4a3573dbe7b64ed89328b42bd5f4d77069514436f715c041ee52ccbe5e5ad"
Oct 30 22:58:03.840050 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.840005    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfd4a3573dbe7b64ed89328b42bd5f4d77069514436f715c041ee52ccbe5e5ad"} err="failed to get container status \"bfd4a3573dbe7b64ed89328b42bd5f4d77069514436f715c041ee52ccbe5e5ad\": rpc error: code = NotFound desc = could not find container \"bfd4a3573dbe7b64ed89328b42bd5f4d77069514436f715c041ee52ccbe5e5ad\": container with ID starting with bfd4a3573dbe7b64ed89328b42bd5f4d77069514436f715c041ee52ccbe5e5ad not found: ID does not exist"
Oct 30 22:58:03.840050 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.840026    5896 scope.go:122] "RemoveContainer" containerID="68e353f8bdc824dbbee9d0f1b9ffd210b00efdc78fb468af21637b84b4d99643"
Oct 30 22:58:03.840273 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:58:03.840252    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68e353f8bdc824dbbee9d0f1b9ffd210b00efdc78fb468af21637b84b4d99643\": container with ID starting with 68e353f8bdc824dbbee9d0f1b9ffd210b00efdc78fb468af21637b84b4d99643 not found: ID does not exist" containerID="68e353f8bdc824dbbee9d0f1b9ffd210b00efdc78fb468af21637b84b4d99643"
Oct 30 22:58:03.840375 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.840276    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68e353f8bdc824dbbee9d0f1b9ffd210b00efdc78fb468af21637b84b4d99643"} err="failed to get container status \"68e353f8bdc824dbbee9d0f1b9ffd210b00efdc78fb468af21637b84b4d99643\": rpc error: code = NotFound desc = could not find container \"68e353f8bdc824dbbee9d0f1b9ffd210b00efdc78fb468af21637b84b4d99643\": container with ID starting with 68e353f8bdc824dbbee9d0f1b9ffd210b00efdc78fb468af21637b84b4d99643 not found: ID does not exist"
Oct 30 22:58:03.840375 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.840292    5896 scope.go:122] "RemoveContainer" containerID="241000e857eef725b6a524452959d2bdffc87ff0c4b15e2c0bf342cba7e58790"
Oct 30 22:58:03.840539 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:58:03.840522    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"241000e857eef725b6a524452959d2bdffc87ff0c4b15e2c0bf342cba7e58790\": container with ID starting with 241000e857eef725b6a524452959d2bdffc87ff0c4b15e2c0bf342cba7e58790 not found: ID does not exist" containerID="241000e857eef725b6a524452959d2bdffc87ff0c4b15e2c0bf342cba7e58790"
Oct 30 22:58:03.840633 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.840548    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"241000e857eef725b6a524452959d2bdffc87ff0c4b15e2c0bf342cba7e58790"} err="failed to get container status \"241000e857eef725b6a524452959d2bdffc87ff0c4b15e2c0bf342cba7e58790\": rpc error: code = NotFound desc = could not find container \"241000e857eef725b6a524452959d2bdffc87ff0c4b15e2c0bf342cba7e58790\": container with ID starting with 241000e857eef725b6a524452959d2bdffc87ff0c4b15e2c0bf342cba7e58790 not found: ID does not exist"
Oct 30 22:58:03.987472 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.987442    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"]
Oct 30 22:58:03.992787 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:03.992766    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-rollback-predictor-5597787b4-pc4ht"]
Oct 30 22:58:05.735842 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:05.735811    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" path="/var/lib/kubelet/pods/9b594ed4-bdc1-4d5b-a2b4-2d6576cced36/volumes"
Oct 30 22:58:08.741307 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741275    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"]
Oct 30 22:58:08.741705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741599    5896 cpu_manager.go:414] "containerMap: removing container" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="storage-initializer"
Oct 30 22:58:08.741705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741610    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="storage-initializer"
Oct 30 22:58:08.741705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741624    5896 cpu_manager.go:414] "containerMap: removing container" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="storage-initializer"
Oct 30 22:58:08.741705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741629    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="storage-initializer"
Oct 30 22:58:08.741705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741658    5896 cpu_manager.go:414] "containerMap: removing container" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kube-rbac-proxy"
Oct 30 22:58:08.741705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741664    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kube-rbac-proxy"
Oct 30 22:58:08.741705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741671    5896 cpu_manager.go:414] "containerMap: removing container" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="storage-initializer"
Oct 30 22:58:08.741705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741676    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="storage-initializer"
Oct 30 22:58:08.741705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741685    5896 cpu_manager.go:414] "containerMap: removing container" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kserve-container"
Oct 30 22:58:08.741705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741690    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kserve-container"
Oct 30 22:58:08.741705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741696    5896 cpu_manager.go:414] "containerMap: removing container" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kube-rbac-proxy"
Oct 30 22:58:08.741705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741701    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kube-rbac-proxy"
Oct 30 22:58:08.741705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741707    5896 cpu_manager.go:414] "containerMap: removing container" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kube-rbac-proxy"
Oct 30 22:58:08.741705 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741712    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kube-rbac-proxy"
Oct 30 22:58:08.742123 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741730    5896 cpu_manager.go:414] "containerMap: removing container" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kserve-container"
Oct 30 22:58:08.742123 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741735    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kserve-container"
Oct 30 22:58:08.742123 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741745    5896 cpu_manager.go:414] "containerMap: removing container" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kserve-container"
Oct 30 22:58:08.742123 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741749    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kserve-container"
Oct 30 22:58:08.742123 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741810    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kserve-container"
Oct 30 22:58:08.742123 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741818    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kserve-container"
Oct 30 22:58:08.742123 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741827    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4258393-5668-45c3-aca9-e3517715ec6b" containerName="kube-rbac-proxy"
Oct 30 22:58:08.742123 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741833    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kserve-container"
Oct 30 22:58:08.742123 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741837    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="171e1f63-a52e-4c46-8d55-3c11f365f623" containerName="kube-rbac-proxy"
Oct 30 22:58:08.742123 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.741842    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b594ed4-bdc1-4d5b-a2b4-2d6576cced36" containerName="kube-rbac-proxy"
Oct 30 22:58:08.746252 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.746235    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:08.802222 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.802195    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-canary-stop-kube-rbac-proxy-sar-config\""
Oct 30 22:58:08.851721 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.851699    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"seaweedfs-s3-creds\""
Oct 30 22:58:08.851976 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.851958    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"kube-root-ca.crt\""
Oct 30 22:58:08.852085 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.851988    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"default-dockercfg-s9cqw\""
Oct 30 22:58:08.852085 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.852003    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"openshift-service-ca.crt\""
Oct 30 22:58:08.852190 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.852135    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-canary-stop-predictor-serving-cert\""
Oct 30 22:58:08.852190 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.852148    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:58:08.853429 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.853413    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"]
Oct 30 22:58:08.862380 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.862361    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kserve-provision-location\") pod \"isvc-canary-stop-predictor-c7bb5549b-z8dkt\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") " pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:08.862444 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.862388    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e29a8fe9-9485-4c37-b7b8-de41a491e23e-proxy-tls\") pod \"isvc-canary-stop-predictor-c7bb5549b-z8dkt\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") " pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:08.862486 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.862446    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghswd\" (UniqueName: \"kubernetes.io/projected/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kube-api-access-ghswd\") pod \"isvc-canary-stop-predictor-c7bb5549b-z8dkt\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") " pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:08.862528 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.862502    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e29a8fe9-9485-4c37-b7b8-de41a491e23e-cabundle-cert\") pod \"isvc-canary-stop-predictor-c7bb5549b-z8dkt\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") " pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:08.862563 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.862535    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-c7bb5549b-z8dkt\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") " pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:08.963020 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.962994    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kserve-provision-location\") pod \"isvc-canary-stop-predictor-c7bb5549b-z8dkt\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") " pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:08.963138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.963025    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e29a8fe9-9485-4c37-b7b8-de41a491e23e-proxy-tls\") pod \"isvc-canary-stop-predictor-c7bb5549b-z8dkt\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") " pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:08.963138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.963065    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghswd\" (UniqueName: \"kubernetes.io/projected/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kube-api-access-ghswd\") pod \"isvc-canary-stop-predictor-c7bb5549b-z8dkt\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") " pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:08.963138 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.963099    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e29a8fe9-9485-4c37-b7b8-de41a491e23e-cabundle-cert\") pod \"isvc-canary-stop-predictor-c7bb5549b-z8dkt\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") " pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:08.963299 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.963149    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-c7bb5549b-z8dkt\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") " pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:08.963433 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.963410    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kserve-provision-location\") pod \"isvc-canary-stop-predictor-c7bb5549b-z8dkt\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") " pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:08.963786 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.963770    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e29a8fe9-9485-4c37-b7b8-de41a491e23e-cabundle-cert\") pod \"isvc-canary-stop-predictor-c7bb5549b-z8dkt\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") " pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:08.963833 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.963792    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-c7bb5549b-z8dkt\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") " pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:08.965602 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:08.965579    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e29a8fe9-9485-4c37-b7b8-de41a491e23e-proxy-tls\") pod \"isvc-canary-stop-predictor-c7bb5549b-z8dkt\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") " pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:09.026545 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:09.026487    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghswd\" (UniqueName: \"kubernetes.io/projected/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kube-api-access-ghswd\") pod \"isvc-canary-stop-predictor-c7bb5549b-z8dkt\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") " pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:09.062442 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:09.062421    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:09.485223 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:09.485150    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"]
Oct 30 22:58:09.487605 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:58:09.487579    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode29a8fe9_9485_4c37_b7b8_de41a491e23e.slice/crio-a108e33d28d3acf84e2703868d3e154153c9bed25c853109fdd7b452e81c41fa WatchSource:0}: Error finding container a108e33d28d3acf84e2703868d3e154153c9bed25c853109fdd7b452e81c41fa: Status 404 returned error can't find the container with id a108e33d28d3acf84e2703868d3e154153c9bed25c853109fdd7b452e81c41fa
Oct 30 22:58:09.845060 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:09.844981    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" event={"ID":"e29a8fe9-9485-4c37-b7b8-de41a491e23e","Type":"ContainerStarted","Data":"9b99979ef025994a086df85b69ed2ff79e40d22281ae491fa75eccc87b8b0c2b"}
Oct 30 22:58:09.845060 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:09.845045    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" event={"ID":"e29a8fe9-9485-4c37-b7b8-de41a491e23e","Type":"ContainerStarted","Data":"a108e33d28d3acf84e2703868d3e154153c9bed25c853109fdd7b452e81c41fa"}
Oct 30 22:58:13.865704 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:13.865609    5896 generic.go:356] "Generic (PLEG): container finished" podID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerID="9b99979ef025994a086df85b69ed2ff79e40d22281ae491fa75eccc87b8b0c2b" exitCode=0
Oct 30 22:58:13.865704 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:13.865671    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" event={"ID":"e29a8fe9-9485-4c37-b7b8-de41a491e23e","Type":"ContainerDied","Data":"9b99979ef025994a086df85b69ed2ff79e40d22281ae491fa75eccc87b8b0c2b"}
Oct 30 22:58:14.872725 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:14.872685    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" event={"ID":"e29a8fe9-9485-4c37-b7b8-de41a491e23e","Type":"ContainerStarted","Data":"406b04e808a0d2326e1f1f8752f1ffeaa9c1c58b7c708452697546dfaf70801f"}
Oct 30 22:58:14.873154 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:14.872769    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" event={"ID":"e29a8fe9-9485-4c37-b7b8-de41a491e23e","Type":"ContainerStarted","Data":"7cad02958bea833bb1fa169f602aeefd94177235d637ba47247ad5275c501ce6"}
Oct 30 22:58:14.873154 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:14.873052    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:14.873154 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:14.873099    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:14.874112 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:14.874086    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Oct 30 22:58:14.999742 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:14.999629    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" podStartSLOduration=3.069303672 podStartE2EDuration="6.999613278s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.930309606s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:58:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:58:14.999152719 +0000 UTC m=+2947.410618722" watchObservedRunningTime="2026-10-30 22:58:14.999613278 +0000 UTC m=+2947.411079247"
Oct 30 22:58:15.877787 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:15.877744    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Oct 30 22:58:20.882591 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:20.882562    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:58:20.883070 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:20.883043    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Oct 30 22:58:30.883746 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:30.883707    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Oct 30 22:58:32.628515 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:58:32.628480    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:58:40.883058 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:40.883020    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Oct 30 22:58:50.883837 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:58:50.883800    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Oct 30 22:59:00.884037 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:00.883997    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Oct 30 22:59:02.625806 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:59:02.625773    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:59:02.687405 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.687377    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"]
Oct 30 22:59:02.691378 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.691363    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"
Oct 30 22:59:02.785017 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.784993    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-proxy-tls\") pod \"isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd\" (UID: \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"
Oct 30 22:59:02.785128 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.785036    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dlmg\" (UniqueName: \"kubernetes.io/projected/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kube-api-access-9dlmg\") pod \"isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd\" (UID: \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"
Oct 30 22:59:02.785128 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.785111    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd\" (UID: \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"
Oct 30 22:59:02.785205 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.785164    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kserve-provision-location\") pod \"isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd\" (UID: \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"
Oct 30 22:59:02.796257 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.796220    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"storage-config\""
Oct 30 22:59:02.796371 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.796299    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"kube-root-ca.crt\""
Oct 30 22:59:02.796371 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.796301    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-sklearn-s3-tls-global-fail-kube-rbac-proxy-sar-config\""
Oct 30 22:59:02.796371 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.796310    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-sklearn-s3-tls-global-fail-predictor-serving-cert\""
Oct 30 22:59:02.796371 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.796299    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"openshift-service-ca.crt\""
Oct 30 22:59:02.796507 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.796378    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"default-dockercfg-nnb5w\""
Oct 30 22:59:02.797737 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.797715    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"]
Oct 30 22:59:02.885934 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.885878    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kserve-provision-location\") pod \"isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd\" (UID: \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"
Oct 30 22:59:02.885934 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.885917    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-proxy-tls\") pod \"isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd\" (UID: \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"
Oct 30 22:59:02.886082 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.885947    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dlmg\" (UniqueName: \"kubernetes.io/projected/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kube-api-access-9dlmg\") pod \"isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd\" (UID: \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"
Oct 30 22:59:02.886082 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.885985    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd\" (UID: \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"
Oct 30 22:59:02.886255 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.886238    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kserve-provision-location\") pod \"isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd\" (UID: \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"
Oct 30 22:59:02.886620 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.886598    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd\" (UID: \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"
Oct 30 22:59:02.888322 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.888304    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-proxy-tls\") pod \"isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd\" (UID: \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"
Oct 30 22:59:02.950445 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:02.950427    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dlmg\" (UniqueName: \"kubernetes.io/projected/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kube-api-access-9dlmg\") pod \"isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd\" (UID: \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"
Oct 30 22:59:03.008585 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:03.008564    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"
Oct 30 22:59:03.433606 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:03.433584    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"]
Oct 30 22:59:03.435722 ip-10-0-138-202 kubenswrapper[5896]: W1030 22:59:03.435691    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30ca6e1b_c9dd_49f4_b70f_7844d22b1781.slice/crio-f9134abf7029add4b4c7d2e72f0bd7bfbb8750423da1dd01e637ebd04aef6fe2 WatchSource:0}: Error finding container f9134abf7029add4b4c7d2e72f0bd7bfbb8750423da1dd01e637ebd04aef6fe2: Status 404 returned error can't find the container with id f9134abf7029add4b4c7d2e72f0bd7bfbb8750423da1dd01e637ebd04aef6fe2
Oct 30 22:59:04.085136 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:04.085104    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd" event={"ID":"30ca6e1b-c9dd-49f4-b70f-7844d22b1781","Type":"ContainerStarted","Data":"288f077abdc14f450ca6da366a5c6b5ad7110893cd44e0d3ba63cf6859fb424c"}
Oct 30 22:59:04.085513 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:04.085172    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd" event={"ID":"30ca6e1b-c9dd-49f4-b70f-7844d22b1781","Type":"ContainerStarted","Data":"f9134abf7029add4b4c7d2e72f0bd7bfbb8750423da1dd01e637ebd04aef6fe2"}
Oct 30 22:59:08.044662 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:08.044547    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:59:08.055535 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:08.055514    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 22:59:09.107361 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:09.107337    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd_30ca6e1b-c9dd-49f4-b70f-7844d22b1781/storage-initializer/0.log"
Oct 30 22:59:09.107731 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:09.107376    5896 generic.go:356] "Generic (PLEG): container finished" podID="30ca6e1b-c9dd-49f4-b70f-7844d22b1781" containerID="288f077abdc14f450ca6da366a5c6b5ad7110893cd44e0d3ba63cf6859fb424c" exitCode=1
Oct 30 22:59:09.107731 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:09.107456    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd" event={"ID":"30ca6e1b-c9dd-49f4-b70f-7844d22b1781","Type":"ContainerDied","Data":"288f077abdc14f450ca6da366a5c6b5ad7110893cd44e0d3ba63cf6859fb424c"}
Oct 30 22:59:10.112779 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:10.112747    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd_30ca6e1b-c9dd-49f4-b70f-7844d22b1781/storage-initializer/0.log"
Oct 30 22:59:10.113197 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:10.112871    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd" event={"ID":"30ca6e1b-c9dd-49f4-b70f-7844d22b1781","Type":"ContainerStarted","Data":"eed365dcf087add23affc9c219e3e9c382be001870ac7837dfa9b2ef960d64ed"}
Oct 30 22:59:10.883057 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:10.883020    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Oct 30 22:59:12.173155 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:12.173113    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"]
Oct 30 22:59:12.173632 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:12.173314    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd" podUID="30ca6e1b-c9dd-49f4-b70f-7844d22b1781" containerName="storage-initializer" containerID="cri-o://eed365dcf087add23affc9c219e3e9c382be001870ac7837dfa9b2ef960d64ed" gracePeriod=30
Oct 30 22:59:14.010590 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.010568    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd_30ca6e1b-c9dd-49f4-b70f-7844d22b1781/storage-initializer/1.log"
Oct 30 22:59:14.010985 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.010972    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd_30ca6e1b-c9dd-49f4-b70f-7844d22b1781/storage-initializer/0.log"
Oct 30 22:59:14.013245 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.013231    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"
Oct 30 22:59:14.074560 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.074494    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kube-rbac-proxy-sar-config\") pod \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\" (UID: \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\") "
Oct 30 22:59:14.074701 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.074560    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kserve-provision-location\") pod \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\" (UID: \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\") "
Oct 30 22:59:14.074701 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.074608    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-proxy-tls\" (UniqueName: \"kubernetes.io/secret/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-proxy-tls\") pod \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\" (UID: \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\") "
Oct 30 22:59:14.074701 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.074651    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kube-api-access-9dlmg\" (UniqueName: \"kubernetes.io/projected/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kube-api-access-9dlmg\") pod \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\" (UID: \"30ca6e1b-c9dd-49f4-b70f-7844d22b1781\") "
Oct 30 22:59:14.074893 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.074813    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kserve-provision-location" pod "30ca6e1b-c9dd-49f4-b70f-7844d22b1781" (UID: "30ca6e1b-c9dd-49f4-b70f-7844d22b1781"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:59:14.074893 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.074853    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kube-rbac-proxy-sar-config" pod "30ca6e1b-c9dd-49f4-b70f-7844d22b1781" (UID: "30ca6e1b-c9dd-49f4-b70f-7844d22b1781"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:59:14.076684 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.076662    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-proxy-tls" pod "30ca6e1b-c9dd-49f4-b70f-7844d22b1781" (UID: "30ca6e1b-c9dd-49f4-b70f-7844d22b1781"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:59:14.076887 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.076864    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kube-api-access-9dlmg" pod "30ca6e1b-c9dd-49f4-b70f-7844d22b1781" (UID: "30ca6e1b-c9dd-49f4-b70f-7844d22b1781"). InnerVolumeSpecName "kube-api-access-9dlmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:59:14.131932 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.131913    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd_30ca6e1b-c9dd-49f4-b70f-7844d22b1781/storage-initializer/1.log"
Oct 30 22:59:14.132307 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.132295    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd_30ca6e1b-c9dd-49f4-b70f-7844d22b1781/storage-initializer/0.log"
Oct 30 22:59:14.132357 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.132329    5896 generic.go:356] "Generic (PLEG): container finished" podID="30ca6e1b-c9dd-49f4-b70f-7844d22b1781" containerID="eed365dcf087add23affc9c219e3e9c382be001870ac7837dfa9b2ef960d64ed" exitCode=1
Oct 30 22:59:14.132404 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.132390    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"
Oct 30 22:59:14.132439 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.132408    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd" event={"ID":"30ca6e1b-c9dd-49f4-b70f-7844d22b1781","Type":"ContainerDied","Data":"eed365dcf087add23affc9c219e3e9c382be001870ac7837dfa9b2ef960d64ed"}
Oct 30 22:59:14.132483 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.132472    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd" event={"ID":"30ca6e1b-c9dd-49f4-b70f-7844d22b1781","Type":"ContainerDied","Data":"f9134abf7029add4b4c7d2e72f0bd7bfbb8750423da1dd01e637ebd04aef6fe2"}
Oct 30 22:59:14.132518 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.132502    5896 scope.go:122] "RemoveContainer" containerID="eed365dcf087add23affc9c219e3e9c382be001870ac7837dfa9b2ef960d64ed"
Oct 30 22:59:14.140691 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.140673    5896 scope.go:122] "RemoveContainer" containerID="288f077abdc14f450ca6da366a5c6b5ad7110893cd44e0d3ba63cf6859fb424c"
Oct 30 22:59:14.147835 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.147818    5896 scope.go:122] "RemoveContainer" containerID="eed365dcf087add23affc9c219e3e9c382be001870ac7837dfa9b2ef960d64ed"
Oct 30 22:59:14.148115 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:59:14.148098    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eed365dcf087add23affc9c219e3e9c382be001870ac7837dfa9b2ef960d64ed\": container with ID starting with eed365dcf087add23affc9c219e3e9c382be001870ac7837dfa9b2ef960d64ed not found: ID does not exist" containerID="eed365dcf087add23affc9c219e3e9c382be001870ac7837dfa9b2ef960d64ed"
Oct 30 22:59:14.148163 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.148123    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eed365dcf087add23affc9c219e3e9c382be001870ac7837dfa9b2ef960d64ed"} err="failed to get container status \"eed365dcf087add23affc9c219e3e9c382be001870ac7837dfa9b2ef960d64ed\": rpc error: code = NotFound desc = could not find container \"eed365dcf087add23affc9c219e3e9c382be001870ac7837dfa9b2ef960d64ed\": container with ID starting with eed365dcf087add23affc9c219e3e9c382be001870ac7837dfa9b2ef960d64ed not found: ID does not exist"
Oct 30 22:59:14.148163 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.148140    5896 scope.go:122] "RemoveContainer" containerID="288f077abdc14f450ca6da366a5c6b5ad7110893cd44e0d3ba63cf6859fb424c"
Oct 30 22:59:14.148369 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:59:14.148351    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"288f077abdc14f450ca6da366a5c6b5ad7110893cd44e0d3ba63cf6859fb424c\": container with ID starting with 288f077abdc14f450ca6da366a5c6b5ad7110893cd44e0d3ba63cf6859fb424c not found: ID does not exist" containerID="288f077abdc14f450ca6da366a5c6b5ad7110893cd44e0d3ba63cf6859fb424c"
Oct 30 22:59:14.148409 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.148376    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288f077abdc14f450ca6da366a5c6b5ad7110893cd44e0d3ba63cf6859fb424c"} err="failed to get container status \"288f077abdc14f450ca6da366a5c6b5ad7110893cd44e0d3ba63cf6859fb424c\": rpc error: code = NotFound desc = could not find container \"288f077abdc14f450ca6da366a5c6b5ad7110893cd44e0d3ba63cf6859fb424c\": container with ID starting with 288f077abdc14f450ca6da366a5c6b5ad7110893cd44e0d3ba63cf6859fb424c not found: ID does not exist"
Oct 30 22:59:14.176148 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.176126    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:59:14.176148 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.176146    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:59:14.176266 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.176157    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:59:14.176266 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.176166    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dlmg\" (UniqueName: \"kubernetes.io/projected/30ca6e1b-c9dd-49f4-b70f-7844d22b1781-kube-api-access-9dlmg\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:59:14.420669 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.420575    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"]
Oct 30 22:59:14.425807 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:14.425785    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-global-fail-predictor-7dd6fb7f5d-8qlsd"]
Oct 30 22:59:15.735545 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:15.735512    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30ca6e1b-c9dd-49f4-b70f-7844d22b1781" path="/var/lib/kubelet/pods/30ca6e1b-c9dd-49f4-b70f-7844d22b1781/volumes"
Oct 30 22:59:20.883550 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:20.883513    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Oct 30 22:59:22.731433 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:22.731395    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Oct 30 22:59:32.628947 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:59:32.628911    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 22:59:32.731823 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:32.731797    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:59:48.257156 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:48.257120    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"]
Oct 30 22:59:48.257553 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:48.257363    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kserve-container" containerID="cri-o://7cad02958bea833bb1fa169f602aeefd94177235d637ba47247ad5275c501ce6" gracePeriod=30
Oct 30 22:59:48.257553 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:48.257395    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kube-rbac-proxy" containerID="cri-o://406b04e808a0d2326e1f1f8752f1ffeaa9c1c58b7c708452697546dfaf70801f" gracePeriod=30
Oct 30 22:59:49.294499 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:49.294462    5896 generic.go:356] "Generic (PLEG): container finished" podID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerID="406b04e808a0d2326e1f1f8752f1ffeaa9c1c58b7c708452697546dfaf70801f" exitCode=0
Oct 30 22:59:49.294905 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:49.294510    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" event={"ID":"e29a8fe9-9485-4c37-b7b8-de41a491e23e","Type":"ContainerDied","Data":"406b04e808a0d2326e1f1f8752f1ffeaa9c1c58b7c708452697546dfaf70801f"}
Oct 30 22:59:50.878865 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:50.878818    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.36:8643/healthz\": dial tcp 10.134.0.36:8643: connect: connection refused"
Oct 30 22:59:52.597245 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:52.597225    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:59:52.773284 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:52.773256    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kube-api-access-ghswd\" (UniqueName: \"kubernetes.io/projected/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kube-api-access-ghswd\") pod \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") "
Oct 30 22:59:52.773446 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:52.773298    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kube-rbac-proxy-sar-config\") pod \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") "
Oct 30 22:59:52.773446 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:52.773343    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kserve-provision-location\") pod \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") "
Oct 30 22:59:52.773446 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:52.773381    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e29a8fe9-9485-4c37-b7b8-de41a491e23e-proxy-tls\" (UniqueName: \"kubernetes.io/secret/e29a8fe9-9485-4c37-b7b8-de41a491e23e-proxy-tls\") pod \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") "
Oct 30 22:59:52.773610 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:52.773444    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e29a8fe9-9485-4c37-b7b8-de41a491e23e-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e29a8fe9-9485-4c37-b7b8-de41a491e23e-cabundle-cert\") pod \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\" (UID: \"e29a8fe9-9485-4c37-b7b8-de41a491e23e\") "
Oct 30 22:59:52.773762 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:52.773725    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kserve-provision-location" pod "e29a8fe9-9485-4c37-b7b8-de41a491e23e" (UID: "e29a8fe9-9485-4c37-b7b8-de41a491e23e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:59:52.773762 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:52.773741    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kube-rbac-proxy-sar-config" pod "e29a8fe9-9485-4c37-b7b8-de41a491e23e" (UID: "e29a8fe9-9485-4c37-b7b8-de41a491e23e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:59:52.773762 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:52.773760    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e29a8fe9-9485-4c37-b7b8-de41a491e23e-cabundle-cert" pod "e29a8fe9-9485-4c37-b7b8-de41a491e23e" (UID: "e29a8fe9-9485-4c37-b7b8-de41a491e23e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:59:52.775613 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:52.775588    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kube-api-access-ghswd" pod "e29a8fe9-9485-4c37-b7b8-de41a491e23e" (UID: "e29a8fe9-9485-4c37-b7b8-de41a491e23e"). InnerVolumeSpecName "kube-api-access-ghswd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:59:52.775793 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:52.775681    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e29a8fe9-9485-4c37-b7b8-de41a491e23e-proxy-tls" pod "e29a8fe9-9485-4c37-b7b8-de41a491e23e" (UID: "e29a8fe9-9485-4c37-b7b8-de41a491e23e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:59:52.874868 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:52.874841    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghswd\" (UniqueName: \"kubernetes.io/projected/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kube-api-access-ghswd\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:59:52.874868 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:52.874868    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:59:52.875007 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:52.874878    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e29a8fe9-9485-4c37-b7b8-de41a491e23e-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:59:52.875007 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:52.874887    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e29a8fe9-9485-4c37-b7b8-de41a491e23e-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:59:52.875007 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:52.874896    5896 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e29a8fe9-9485-4c37-b7b8-de41a491e23e-cabundle-cert\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:59:53.314901 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:53.314868    5896 generic.go:356] "Generic (PLEG): container finished" podID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerID="7cad02958bea833bb1fa169f602aeefd94177235d637ba47247ad5275c501ce6" exitCode=0
Oct 30 22:59:53.315063 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:53.314927    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"
Oct 30 22:59:53.315063 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:53.314947    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" event={"ID":"e29a8fe9-9485-4c37-b7b8-de41a491e23e","Type":"ContainerDied","Data":"7cad02958bea833bb1fa169f602aeefd94177235d637ba47247ad5275c501ce6"}
Oct 30 22:59:53.315063 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:53.315006    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt" event={"ID":"e29a8fe9-9485-4c37-b7b8-de41a491e23e","Type":"ContainerDied","Data":"a108e33d28d3acf84e2703868d3e154153c9bed25c853109fdd7b452e81c41fa"}
Oct 30 22:59:53.315063 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:53.315036    5896 scope.go:122] "RemoveContainer" containerID="406b04e808a0d2326e1f1f8752f1ffeaa9c1c58b7c708452697546dfaf70801f"
Oct 30 22:59:53.323843 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:53.323827    5896 scope.go:122] "RemoveContainer" containerID="7cad02958bea833bb1fa169f602aeefd94177235d637ba47247ad5275c501ce6"
Oct 30 22:59:53.331239 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:53.331224    5896 scope.go:122] "RemoveContainer" containerID="9b99979ef025994a086df85b69ed2ff79e40d22281ae491fa75eccc87b8b0c2b"
Oct 30 22:59:53.338896 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:53.338881    5896 scope.go:122] "RemoveContainer" containerID="406b04e808a0d2326e1f1f8752f1ffeaa9c1c58b7c708452697546dfaf70801f"
Oct 30 22:59:53.339154 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:59:53.339138    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"406b04e808a0d2326e1f1f8752f1ffeaa9c1c58b7c708452697546dfaf70801f\": container with ID starting with 406b04e808a0d2326e1f1f8752f1ffeaa9c1c58b7c708452697546dfaf70801f not found: ID does not exist" containerID="406b04e808a0d2326e1f1f8752f1ffeaa9c1c58b7c708452697546dfaf70801f"
Oct 30 22:59:53.339201 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:53.339161    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"406b04e808a0d2326e1f1f8752f1ffeaa9c1c58b7c708452697546dfaf70801f"} err="failed to get container status \"406b04e808a0d2326e1f1f8752f1ffeaa9c1c58b7c708452697546dfaf70801f\": rpc error: code = NotFound desc = could not find container \"406b04e808a0d2326e1f1f8752f1ffeaa9c1c58b7c708452697546dfaf70801f\": container with ID starting with 406b04e808a0d2326e1f1f8752f1ffeaa9c1c58b7c708452697546dfaf70801f not found: ID does not exist"
Oct 30 22:59:53.339201 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:53.339179    5896 scope.go:122] "RemoveContainer" containerID="7cad02958bea833bb1fa169f602aeefd94177235d637ba47247ad5275c501ce6"
Oct 30 22:59:53.339377 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:59:53.339364    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cad02958bea833bb1fa169f602aeefd94177235d637ba47247ad5275c501ce6\": container with ID starting with 7cad02958bea833bb1fa169f602aeefd94177235d637ba47247ad5275c501ce6 not found: ID does not exist" containerID="7cad02958bea833bb1fa169f602aeefd94177235d637ba47247ad5275c501ce6"
Oct 30 22:59:53.339414 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:53.339381    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cad02958bea833bb1fa169f602aeefd94177235d637ba47247ad5275c501ce6"} err="failed to get container status \"7cad02958bea833bb1fa169f602aeefd94177235d637ba47247ad5275c501ce6\": rpc error: code = NotFound desc = could not find container \"7cad02958bea833bb1fa169f602aeefd94177235d637ba47247ad5275c501ce6\": container with ID starting with 7cad02958bea833bb1fa169f602aeefd94177235d637ba47247ad5275c501ce6 not found: ID does not exist"
Oct 30 22:59:53.339414 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:53.339392    5896 scope.go:122] "RemoveContainer" containerID="9b99979ef025994a086df85b69ed2ff79e40d22281ae491fa75eccc87b8b0c2b"
Oct 30 22:59:53.339569 ip-10-0-138-202 kubenswrapper[5896]: E1030 22:59:53.339555    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b99979ef025994a086df85b69ed2ff79e40d22281ae491fa75eccc87b8b0c2b\": container with ID starting with 9b99979ef025994a086df85b69ed2ff79e40d22281ae491fa75eccc87b8b0c2b not found: ID does not exist" containerID="9b99979ef025994a086df85b69ed2ff79e40d22281ae491fa75eccc87b8b0c2b"
Oct 30 22:59:53.339609 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:53.339571    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b99979ef025994a086df85b69ed2ff79e40d22281ae491fa75eccc87b8b0c2b"} err="failed to get container status \"9b99979ef025994a086df85b69ed2ff79e40d22281ae491fa75eccc87b8b0c2b\": rpc error: code = NotFound desc = could not find container \"9b99979ef025994a086df85b69ed2ff79e40d22281ae491fa75eccc87b8b0c2b\": container with ID starting with 9b99979ef025994a086df85b69ed2ff79e40d22281ae491fa75eccc87b8b0c2b not found: ID does not exist"
Oct 30 22:59:53.484544 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:53.484520    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"]
Oct 30 22:59:53.535116 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:53.535095    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-959ee2f135/isvc-canary-stop-predictor-c7bb5549b-z8dkt"]
Oct 30 22:59:53.735969 ip-10-0-138-202 kubenswrapper[5896]: I1030 22:59:53.735944    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" path="/var/lib/kubelet/pods/e29a8fe9-9485-4c37-b7b8-de41a491e23e/volumes"
Oct 30 23:00:02.624869 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:00:02.624830    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:00:32.627183 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:00:32.627096    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:00:49.079357 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.079322    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"]
Oct 30 23:00:49.079802 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.079660    5896 cpu_manager.go:414] "containerMap: removing container" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="storage-initializer"
Oct 30 23:00:49.079802 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.079676    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="storage-initializer"
Oct 30 23:00:49.079802 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.079689    5896 cpu_manager.go:414] "containerMap: removing container" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kserve-container"
Oct 30 23:00:49.079802 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.079694    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kserve-container"
Oct 30 23:00:49.079802 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.079702    5896 cpu_manager.go:414] "containerMap: removing container" podUID="30ca6e1b-c9dd-49f4-b70f-7844d22b1781" containerName="storage-initializer"
Oct 30 23:00:49.079802 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.079708    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30ca6e1b-c9dd-49f4-b70f-7844d22b1781" containerName="storage-initializer"
Oct 30 23:00:49.079802 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.079716    5896 cpu_manager.go:414] "containerMap: removing container" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kube-rbac-proxy"
Oct 30 23:00:49.079802 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.079721    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kube-rbac-proxy"
Oct 30 23:00:49.079802 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.079732    5896 cpu_manager.go:414] "containerMap: removing container" podUID="30ca6e1b-c9dd-49f4-b70f-7844d22b1781" containerName="storage-initializer"
Oct 30 23:00:49.079802 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.079736    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30ca6e1b-c9dd-49f4-b70f-7844d22b1781" containerName="storage-initializer"
Oct 30 23:00:49.079802 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.079793    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kserve-container"
Oct 30 23:00:49.079802 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.079800    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="30ca6e1b-c9dd-49f4-b70f-7844d22b1781" containerName="storage-initializer"
Oct 30 23:00:49.079802 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.079806    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="e29a8fe9-9485-4c37-b7b8-de41a491e23e" containerName="kube-rbac-proxy"
Oct 30 23:00:49.080246 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.079813    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="30ca6e1b-c9dd-49f4-b70f-7844d22b1781" containerName="storage-initializer"
Oct 30 23:00:49.082844 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.082829    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"
Oct 30 23:00:49.137748 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.137725    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"kube-root-ca.crt\""
Oct 30 23:00:49.183367 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.183341    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/621491d5-6be3-4575-89a5-be2dc29443aa-proxy-tls\") pod \"isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h\" (UID: \"621491d5-6be3-4575-89a5-be2dc29443aa\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"
Oct 30 23:00:49.183492 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.183379    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j895r\" (UniqueName: \"kubernetes.io/projected/621491d5-6be3-4575-89a5-be2dc29443aa-kube-api-access-j895r\") pod \"isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h\" (UID: \"621491d5-6be3-4575-89a5-be2dc29443aa\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"
Oct 30 23:00:49.183492 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.183469    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/621491d5-6be3-4575-89a5-be2dc29443aa-kserve-provision-location\") pod \"isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h\" (UID: \"621491d5-6be3-4575-89a5-be2dc29443aa\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"
Oct 30 23:00:49.183716 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.183581    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/621491d5-6be3-4575-89a5-be2dc29443aa-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h\" (UID: \"621491d5-6be3-4575-89a5-be2dc29443aa\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"
Oct 30 23:00:49.187408 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.187384    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"storage-config\""
Oct 30 23:00:49.187507 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.187462    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-sklearn-s3-tls-custom-fail-kube-rbac-proxy-sar-config\""
Oct 30 23:00:49.187507 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.187475    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-sklearn-s3-tls-custom-fail-predictor-serving-cert\""
Oct 30 23:00:49.187599 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.187511    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"openshift-service-ca.crt\""
Oct 30 23:00:49.187599 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.187552    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"default-dockercfg-nnb5w\""
Oct 30 23:00:49.189012 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.188989    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"]
Oct 30 23:00:49.284651 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.284622    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/621491d5-6be3-4575-89a5-be2dc29443aa-proxy-tls\") pod \"isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h\" (UID: \"621491d5-6be3-4575-89a5-be2dc29443aa\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"
Oct 30 23:00:49.284760 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.284662    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j895r\" (UniqueName: \"kubernetes.io/projected/621491d5-6be3-4575-89a5-be2dc29443aa-kube-api-access-j895r\") pod \"isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h\" (UID: \"621491d5-6be3-4575-89a5-be2dc29443aa\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"
Oct 30 23:00:49.284760 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.284703    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/621491d5-6be3-4575-89a5-be2dc29443aa-kserve-provision-location\") pod \"isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h\" (UID: \"621491d5-6be3-4575-89a5-be2dc29443aa\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"
Oct 30 23:00:49.284760 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.284719    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/621491d5-6be3-4575-89a5-be2dc29443aa-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h\" (UID: \"621491d5-6be3-4575-89a5-be2dc29443aa\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"
Oct 30 23:00:49.285098 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.285078    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/621491d5-6be3-4575-89a5-be2dc29443aa-kserve-provision-location\") pod \"isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h\" (UID: \"621491d5-6be3-4575-89a5-be2dc29443aa\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"
Oct 30 23:00:49.285404 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.285388    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/621491d5-6be3-4575-89a5-be2dc29443aa-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h\" (UID: \"621491d5-6be3-4575-89a5-be2dc29443aa\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"
Oct 30 23:00:49.287225 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.287206    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/621491d5-6be3-4575-89a5-be2dc29443aa-proxy-tls\") pod \"isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h\" (UID: \"621491d5-6be3-4575-89a5-be2dc29443aa\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"
Oct 30 23:00:49.347943 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.347876    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j895r\" (UniqueName: \"kubernetes.io/projected/621491d5-6be3-4575-89a5-be2dc29443aa-kube-api-access-j895r\") pod \"isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h\" (UID: \"621491d5-6be3-4575-89a5-be2dc29443aa\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"
Oct 30 23:00:49.399878 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.399857    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"
Oct 30 23:00:49.820357 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:49.820335    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"]
Oct 30 23:00:49.822478 ip-10-0-138-202 kubenswrapper[5896]: W1030 23:00:49.822448    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod621491d5_6be3_4575_89a5_be2dc29443aa.slice/crio-3218e29f9e0a6b3f19008a0a179deb795eeff0e73e98dc1f8c658f7fcfe2f0b3 WatchSource:0}: Error finding container 3218e29f9e0a6b3f19008a0a179deb795eeff0e73e98dc1f8c658f7fcfe2f0b3: Status 404 returned error can't find the container with id 3218e29f9e0a6b3f19008a0a179deb795eeff0e73e98dc1f8c658f7fcfe2f0b3
Oct 30 23:00:50.563673 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:50.563616    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h" event={"ID":"621491d5-6be3-4575-89a5-be2dc29443aa","Type":"ContainerStarted","Data":"68a57f81a77efa8162a0e0447ff31e3533809771716f95a9671ba4856202c420"}
Oct 30 23:00:50.564073 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:50.563707    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h" event={"ID":"621491d5-6be3-4575-89a5-be2dc29443aa","Type":"ContainerStarted","Data":"3218e29f9e0a6b3f19008a0a179deb795eeff0e73e98dc1f8c658f7fcfe2f0b3"}
Oct 30 23:00:52.573463 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:52.573437    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h_621491d5-6be3-4575-89a5-be2dc29443aa/storage-initializer/0.log"
Oct 30 23:00:52.573853 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:52.573474    5896 generic.go:356] "Generic (PLEG): container finished" podID="621491d5-6be3-4575-89a5-be2dc29443aa" containerID="68a57f81a77efa8162a0e0447ff31e3533809771716f95a9671ba4856202c420" exitCode=1
Oct 30 23:00:52.573853 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:52.573553    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h" event={"ID":"621491d5-6be3-4575-89a5-be2dc29443aa","Type":"ContainerDied","Data":"68a57f81a77efa8162a0e0447ff31e3533809771716f95a9671ba4856202c420"}
Oct 30 23:00:53.579178 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:53.579153    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h_621491d5-6be3-4575-89a5-be2dc29443aa/storage-initializer/0.log"
Oct 30 23:00:53.579547 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:53.579277    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h" event={"ID":"621491d5-6be3-4575-89a5-be2dc29443aa","Type":"ContainerStarted","Data":"9af35af8cb811d034c6b5d43c4624254576173648262cd1543e94b246af40b88"}
Oct 30 23:00:56.594948 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:56.594922    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h_621491d5-6be3-4575-89a5-be2dc29443aa/storage-initializer/1.log"
Oct 30 23:00:56.595401 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:56.595372    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h_621491d5-6be3-4575-89a5-be2dc29443aa/storage-initializer/0.log"
Oct 30 23:00:56.595445 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:56.595402    5896 generic.go:356] "Generic (PLEG): container finished" podID="621491d5-6be3-4575-89a5-be2dc29443aa" containerID="9af35af8cb811d034c6b5d43c4624254576173648262cd1543e94b246af40b88" exitCode=1
Oct 30 23:00:56.595506 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:56.595481    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h" event={"ID":"621491d5-6be3-4575-89a5-be2dc29443aa","Type":"ContainerDied","Data":"9af35af8cb811d034c6b5d43c4624254576173648262cd1543e94b246af40b88"}
Oct 30 23:00:56.595565 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:56.595554    5896 scope.go:122] "RemoveContainer" containerID="68a57f81a77efa8162a0e0447ff31e3533809771716f95a9671ba4856202c420"
Oct 30 23:00:56.595892 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:56.595875    5896 scope.go:122] "RemoveContainer" containerID="68a57f81a77efa8162a0e0447ff31e3533809771716f95a9671ba4856202c420"
Oct 30 23:00:56.607058 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:00:56.607028    5896 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"storage-initializer\" with CrashLoopBackOff: \"back-off 10s restarting failed container=storage-initializer pod=isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h_e2e-gw1-bc5112d610(621491d5-6be3-4575-89a5-be2dc29443aa)\"" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h" podUID="621491d5-6be3-4575-89a5-be2dc29443aa"
Oct 30 23:00:56.607188 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:00:56.607156    5896 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h_e2e-gw1-bc5112d610_621491d5-6be3-4575-89a5-be2dc29443aa_0 in pod sandbox 3218e29f9e0a6b3f19008a0a179deb795eeff0e73e98dc1f8c658f7fcfe2f0b3 from index: no such id: '68a57f81a77efa8162a0e0447ff31e3533809771716f95a9671ba4856202c420'" containerID="68a57f81a77efa8162a0e0447ff31e3533809771716f95a9671ba4856202c420"
Oct 30 23:00:56.607231 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:56.607202    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68a57f81a77efa8162a0e0447ff31e3533809771716f95a9671ba4856202c420"} err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h_e2e-gw1-bc5112d610_621491d5-6be3-4575-89a5-be2dc29443aa_0 in pod sandbox 3218e29f9e0a6b3f19008a0a179deb795eeff0e73e98dc1f8c658f7fcfe2f0b3 from index: no such id: '68a57f81a77efa8162a0e0447ff31e3533809771716f95a9671ba4856202c420'"
Oct 30 23:00:57.602478 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:57.602444    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h_621491d5-6be3-4575-89a5-be2dc29443aa/storage-initializer/1.log"
Oct 30 23:00:58.902507 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:58.902476    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"]
Oct 30 23:00:59.022824 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.022803    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h_621491d5-6be3-4575-89a5-be2dc29443aa/storage-initializer/1.log"
Oct 30 23:00:59.026226 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.026208    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"
Oct 30 23:00:59.164590 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.164534    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/621491d5-6be3-4575-89a5-be2dc29443aa-proxy-tls\" (UniqueName: \"kubernetes.io/secret/621491d5-6be3-4575-89a5-be2dc29443aa-proxy-tls\") pod \"621491d5-6be3-4575-89a5-be2dc29443aa\" (UID: \"621491d5-6be3-4575-89a5-be2dc29443aa\") "
Oct 30 23:00:59.164590 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.164574    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/621491d5-6be3-4575-89a5-be2dc29443aa-kube-api-access-j895r\" (UniqueName: \"kubernetes.io/projected/621491d5-6be3-4575-89a5-be2dc29443aa-kube-api-access-j895r\") pod \"621491d5-6be3-4575-89a5-be2dc29443aa\" (UID: \"621491d5-6be3-4575-89a5-be2dc29443aa\") "
Oct 30 23:00:59.164765 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.164600    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/621491d5-6be3-4575-89a5-be2dc29443aa-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/621491d5-6be3-4575-89a5-be2dc29443aa-kube-rbac-proxy-sar-config\") pod \"621491d5-6be3-4575-89a5-be2dc29443aa\" (UID: \"621491d5-6be3-4575-89a5-be2dc29443aa\") "
Oct 30 23:00:59.164765 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.164651    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/621491d5-6be3-4575-89a5-be2dc29443aa-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/621491d5-6be3-4575-89a5-be2dc29443aa-kserve-provision-location\") pod \"621491d5-6be3-4575-89a5-be2dc29443aa\" (UID: \"621491d5-6be3-4575-89a5-be2dc29443aa\") "
Oct 30 23:00:59.164973 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.164952    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621491d5-6be3-4575-89a5-be2dc29443aa-kserve-provision-location" pod "621491d5-6be3-4575-89a5-be2dc29443aa" (UID: "621491d5-6be3-4575-89a5-be2dc29443aa"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:00:59.165028 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.164972    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/621491d5-6be3-4575-89a5-be2dc29443aa-kube-rbac-proxy-sar-config" pod "621491d5-6be3-4575-89a5-be2dc29443aa" (UID: "621491d5-6be3-4575-89a5-be2dc29443aa"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:00:59.166862 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.166841    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/621491d5-6be3-4575-89a5-be2dc29443aa-proxy-tls" pod "621491d5-6be3-4575-89a5-be2dc29443aa" (UID: "621491d5-6be3-4575-89a5-be2dc29443aa"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:00:59.166959 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.166871    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/621491d5-6be3-4575-89a5-be2dc29443aa-kube-api-access-j895r" pod "621491d5-6be3-4575-89a5-be2dc29443aa" (UID: "621491d5-6be3-4575-89a5-be2dc29443aa"). InnerVolumeSpecName "kube-api-access-j895r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:00:59.265683 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.265657    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/621491d5-6be3-4575-89a5-be2dc29443aa-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:00:59.265683 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.265679    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j895r\" (UniqueName: \"kubernetes.io/projected/621491d5-6be3-4575-89a5-be2dc29443aa-kube-api-access-j895r\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:00:59.265683 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.265690    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/621491d5-6be3-4575-89a5-be2dc29443aa-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:00:59.265876 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.265699    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/621491d5-6be3-4575-89a5-be2dc29443aa-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:00:59.612530 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.612506    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h_621491d5-6be3-4575-89a5-be2dc29443aa/storage-initializer/1.log"
Oct 30 23:00:59.612731 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.612572    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h" event={"ID":"621491d5-6be3-4575-89a5-be2dc29443aa","Type":"ContainerDied","Data":"3218e29f9e0a6b3f19008a0a179deb795eeff0e73e98dc1f8c658f7fcfe2f0b3"}
Oct 30 23:00:59.612731 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.612604    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"
Oct 30 23:00:59.612731 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.612661    5896 scope.go:122] "RemoveContainer" containerID="9af35af8cb811d034c6b5d43c4624254576173648262cd1543e94b246af40b88"
Oct 30 23:00:59.848382 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.848349    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"]
Oct 30 23:00:59.859810 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:00:59.859779    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-custom-fail-predictor-dfb5c6b7d-m8f6h"]
Oct 30 23:01:01.735713 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:01:01.735680    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="621491d5-6be3-4575-89a5-be2dc29443aa" path="/var/lib/kubelet/pods/621491d5-6be3-4575-89a5-be2dc29443aa/volumes"
Oct 30 23:01:02.627054 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:01:02.627022    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:01:08.118007 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:01:08.117970    5896 scope.go:122] "RemoveContainer" containerID="5d5a8ff14c7f316ac1b7fd39d7118f4cf8b0ab43a005905800614bfebe4a4d7d"
Oct 30 23:01:08.129075 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:01:08.129054    5896 scope.go:122] "RemoveContainer" containerID="00e3aea37e69f8b6e4e161e7bf147c333ed038a4ba2f1c02fc5d02e88fab2a86"
Oct 30 23:01:08.136481 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:01:08.136459    5896 scope.go:122] "RemoveContainer" containerID="67a3b587ff3dd562d0f8ee5895e20b46194c43ebc99874a759472e8b2f115085"
Oct 30 23:01:32.628471 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:01:32.628426    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:02:02.626791 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:02:02.626749    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:02:32.628280 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:02:32.628244    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:02:36.380685 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.380626    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"]
Oct 30 23:02:36.381051 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.380982    5896 cpu_manager.go:414] "containerMap: removing container" podUID="621491d5-6be3-4575-89a5-be2dc29443aa" containerName="storage-initializer"
Oct 30 23:02:36.381051 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.380993    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="621491d5-6be3-4575-89a5-be2dc29443aa" containerName="storage-initializer"
Oct 30 23:02:36.381051 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.381003    5896 cpu_manager.go:414] "containerMap: removing container" podUID="621491d5-6be3-4575-89a5-be2dc29443aa" containerName="storage-initializer"
Oct 30 23:02:36.381051 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.381011    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="621491d5-6be3-4575-89a5-be2dc29443aa" containerName="storage-initializer"
Oct 30 23:02:36.381181 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.381069    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="621491d5-6be3-4575-89a5-be2dc29443aa" containerName="storage-initializer"
Oct 30 23:02:36.381181 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.381078    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="621491d5-6be3-4575-89a5-be2dc29443aa" containerName="storage-initializer"
Oct 30 23:02:36.384068 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.384052    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"
Oct 30 23:02:36.438792 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.438770    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-sklearn-s3-tls-serving-fail-kube-rbac-proxy-sar-config\""
Oct 30 23:02:36.477230 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.477206    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2rg9\" (UniqueName: \"kubernetes.io/projected/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kube-api-access-z2rg9\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2\" (UID: \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"
Oct 30 23:02:36.477330 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.477236    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2\" (UID: \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"
Oct 30 23:02:36.477330 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.477260    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kserve-provision-location\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2\" (UID: \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"
Oct 30 23:02:36.477415 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.477363    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/55eb843f-c309-4127-9bfa-2a8e3e6444fc-proxy-tls\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2\" (UID: \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"
Oct 30 23:02:36.488449 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.488427    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"openshift-service-ca.crt\""
Oct 30 23:02:36.488581 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.488457    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"kube-root-ca.crt\""
Oct 30 23:02:36.488679 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.488652    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"storage-config\""
Oct 30 23:02:36.488679 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.488666    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"default-dockercfg-nnb5w\""
Oct 30 23:02:36.488761 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.488687    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-sklearn-s3-tls-serving-fail-predictor-serving-cert\""
Oct 30 23:02:36.490327 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.490310    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"]
Oct 30 23:02:36.578545 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.578521    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/55eb843f-c309-4127-9bfa-2a8e3e6444fc-proxy-tls\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2\" (UID: \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"
Oct 30 23:02:36.578689 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.578570    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2rg9\" (UniqueName: \"kubernetes.io/projected/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kube-api-access-z2rg9\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2\" (UID: \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"
Oct 30 23:02:36.578689 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.578594    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2\" (UID: \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"
Oct 30 23:02:36.578689 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.578616    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kserve-provision-location\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2\" (UID: \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"
Oct 30 23:02:36.578991 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.578975    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kserve-provision-location\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2\" (UID: \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"
Oct 30 23:02:36.579262 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.579243    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2\" (UID: \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"
Oct 30 23:02:36.581245 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.581228    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/55eb843f-c309-4127-9bfa-2a8e3e6444fc-proxy-tls\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2\" (UID: \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"
Oct 30 23:02:36.641859 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.641810    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2rg9\" (UniqueName: \"kubernetes.io/projected/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kube-api-access-z2rg9\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2\" (UID: \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\") " pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"
Oct 30 23:02:36.700160 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:36.700139    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"
Oct 30 23:02:37.122317 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:37.122293    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"]
Oct 30 23:02:37.124800 ip-10-0-138-202 kubenswrapper[5896]: W1030 23:02:37.124771    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55eb843f_c309_4127_9bfa_2a8e3e6444fc.slice/crio-b63085e87bb2e6a844791fd65886587fad7b70b95da133dfecf4b717983c1bcd WatchSource:0}: Error finding container b63085e87bb2e6a844791fd65886587fad7b70b95da133dfecf4b717983c1bcd: Status 404 returned error can't find the container with id b63085e87bb2e6a844791fd65886587fad7b70b95da133dfecf4b717983c1bcd
Oct 30 23:02:38.006156 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:38.006120    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2" event={"ID":"55eb843f-c309-4127-9bfa-2a8e3e6444fc","Type":"ContainerStarted","Data":"8382d186d8e161d91ff9610068f5f6db8de38dbadfb6f2ef93ed30eb323cb035"}
Oct 30 23:02:38.006559 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:38.006192    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2" event={"ID":"55eb843f-c309-4127-9bfa-2a8e3e6444fc","Type":"ContainerStarted","Data":"b63085e87bb2e6a844791fd65886587fad7b70b95da133dfecf4b717983c1bcd"}
Oct 30 23:02:45.037913 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:45.037885    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2_55eb843f-c309-4127-9bfa-2a8e3e6444fc/storage-initializer/0.log"
Oct 30 23:02:45.038288 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:45.037925    5896 generic.go:356] "Generic (PLEG): container finished" podID="55eb843f-c309-4127-9bfa-2a8e3e6444fc" containerID="8382d186d8e161d91ff9610068f5f6db8de38dbadfb6f2ef93ed30eb323cb035" exitCode=1
Oct 30 23:02:45.038288 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:45.037993    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2" event={"ID":"55eb843f-c309-4127-9bfa-2a8e3e6444fc","Type":"ContainerDied","Data":"8382d186d8e161d91ff9610068f5f6db8de38dbadfb6f2ef93ed30eb323cb035"}
Oct 30 23:02:46.047808 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:46.047781    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2_55eb843f-c309-4127-9bfa-2a8e3e6444fc/storage-initializer/0.log"
Oct 30 23:02:46.048225 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:46.047875    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2" event={"ID":"55eb843f-c309-4127-9bfa-2a8e3e6444fc","Type":"ContainerStarted","Data":"474086e1b1feb457a1628a722af34cd8ed75670da4d541f948b9ef7b89091087"}
Oct 30 23:02:50.065622 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:50.065596    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2_55eb843f-c309-4127-9bfa-2a8e3e6444fc/storage-initializer/1.log"
Oct 30 23:02:50.066021 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:50.065995    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2_55eb843f-c309-4127-9bfa-2a8e3e6444fc/storage-initializer/0.log"
Oct 30 23:02:50.066064 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:50.066025    5896 generic.go:356] "Generic (PLEG): container finished" podID="55eb843f-c309-4127-9bfa-2a8e3e6444fc" containerID="474086e1b1feb457a1628a722af34cd8ed75670da4d541f948b9ef7b89091087" exitCode=1
Oct 30 23:02:50.066098 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:50.066064    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2" event={"ID":"55eb843f-c309-4127-9bfa-2a8e3e6444fc","Type":"ContainerDied","Data":"474086e1b1feb457a1628a722af34cd8ed75670da4d541f948b9ef7b89091087"}
Oct 30 23:02:50.066134 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:50.066117    5896 scope.go:122] "RemoveContainer" containerID="8382d186d8e161d91ff9610068f5f6db8de38dbadfb6f2ef93ed30eb323cb035"
Oct 30 23:02:50.066421 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:50.066399    5896 scope.go:122] "RemoveContainer" containerID="8382d186d8e161d91ff9610068f5f6db8de38dbadfb6f2ef93ed30eb323cb035"
Oct 30 23:02:50.076831 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:02:50.076807    5896 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"storage-initializer\" with CrashLoopBackOff: \"back-off 10s restarting failed container=storage-initializer pod=isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2_e2e-gw1-bc5112d610(55eb843f-c309-4127-9bfa-2a8e3e6444fc)\"" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2" podUID="55eb843f-c309-4127-9bfa-2a8e3e6444fc"
Oct 30 23:02:50.076911 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:02:50.076876    5896 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2_e2e-gw1-bc5112d610_55eb843f-c309-4127-9bfa-2a8e3e6444fc_0 in pod sandbox b63085e87bb2e6a844791fd65886587fad7b70b95da133dfecf4b717983c1bcd from index: no such id: '8382d186d8e161d91ff9610068f5f6db8de38dbadfb6f2ef93ed30eb323cb035'" containerID="8382d186d8e161d91ff9610068f5f6db8de38dbadfb6f2ef93ed30eb323cb035"
Oct 30 23:02:50.076959 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:50.076909    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8382d186d8e161d91ff9610068f5f6db8de38dbadfb6f2ef93ed30eb323cb035"} err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2_e2e-gw1-bc5112d610_55eb843f-c309-4127-9bfa-2a8e3e6444fc_0 in pod sandbox b63085e87bb2e6a844791fd65886587fad7b70b95da133dfecf4b717983c1bcd from index: no such id: '8382d186d8e161d91ff9610068f5f6db8de38dbadfb6f2ef93ed30eb323cb035'"
Oct 30 23:02:51.071324 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:51.071297    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2_55eb843f-c309-4127-9bfa-2a8e3e6444fc/storage-initializer/1.log"
Oct 30 23:02:56.199463 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:56.199423    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"]
Oct 30 23:02:56.329957 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:56.329936    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2_55eb843f-c309-4127-9bfa-2a8e3e6444fc/storage-initializer/1.log"
Oct 30 23:02:56.333266 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:56.333250    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"
Oct 30 23:02:56.426377 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:56.426354    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kube-api-access-z2rg9\" (UniqueName: \"kubernetes.io/projected/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kube-api-access-z2rg9\") pod \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\" (UID: \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\") "
Oct 30 23:02:56.426476 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:56.426414    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kserve-provision-location\") pod \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\" (UID: \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\") "
Oct 30 23:02:56.426476 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:56.426455    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kube-rbac-proxy-sar-config\") pod \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\" (UID: \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\") "
Oct 30 23:02:56.426583 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:56.426485    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55eb843f-c309-4127-9bfa-2a8e3e6444fc-proxy-tls\" (UniqueName: \"kubernetes.io/secret/55eb843f-c309-4127-9bfa-2a8e3e6444fc-proxy-tls\") pod \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\" (UID: \"55eb843f-c309-4127-9bfa-2a8e3e6444fc\") "
Oct 30 23:02:56.426762 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:56.426739    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kserve-provision-location" pod "55eb843f-c309-4127-9bfa-2a8e3e6444fc" (UID: "55eb843f-c309-4127-9bfa-2a8e3e6444fc"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:02:56.426860 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:56.426804    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kube-rbac-proxy-sar-config" pod "55eb843f-c309-4127-9bfa-2a8e3e6444fc" (UID: "55eb843f-c309-4127-9bfa-2a8e3e6444fc"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:02:56.428539 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:56.428521    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kube-api-access-z2rg9" pod "55eb843f-c309-4127-9bfa-2a8e3e6444fc" (UID: "55eb843f-c309-4127-9bfa-2a8e3e6444fc"). InnerVolumeSpecName "kube-api-access-z2rg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:02:56.428609 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:56.428595    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55eb843f-c309-4127-9bfa-2a8e3e6444fc-proxy-tls" pod "55eb843f-c309-4127-9bfa-2a8e3e6444fc" (UID: "55eb843f-c309-4127-9bfa-2a8e3e6444fc"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:02:56.528028 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:56.528007    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:02:56.528028 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:56.528029    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:02:56.528175 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:56.528038    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/55eb843f-c309-4127-9bfa-2a8e3e6444fc-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:02:56.528175 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:56.528047    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2rg9\" (UniqueName: \"kubernetes.io/projected/55eb843f-c309-4127-9bfa-2a8e3e6444fc-kube-api-access-z2rg9\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:02:57.097031 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:57.097002    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-bc5112d610_isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2_55eb843f-c309-4127-9bfa-2a8e3e6444fc/storage-initializer/1.log"
Oct 30 23:02:57.097220 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:57.097116    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"
Oct 30 23:02:57.097220 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:57.097136    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2" event={"ID":"55eb843f-c309-4127-9bfa-2a8e3e6444fc","Type":"ContainerDied","Data":"b63085e87bb2e6a844791fd65886587fad7b70b95da133dfecf4b717983c1bcd"}
Oct 30 23:02:57.097220 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:57.097203    5896 scope.go:122] "RemoveContainer" containerID="474086e1b1feb457a1628a722af34cd8ed75670da4d541f948b9ef7b89091087"
Oct 30 23:02:57.332437 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:57.332407    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"]
Oct 30 23:02:57.335529 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:57.335495    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-bc5112d610/isvc-sklearn-s3-tls-serving-fail-predictor-5cdbcdfc5c-8q7f2"]
Oct 30 23:02:57.735889 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:02:57.735850    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55eb843f-c309-4127-9bfa-2a8e3e6444fc" path="/var/lib/kubelet/pods/55eb843f-c309-4127-9bfa-2a8e3e6444fc/volumes"
Oct 30 23:03:02.626091 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:03:02.626058    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:03:32.628178 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:03:32.628098    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:04:02.625735 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:04:02.625697    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:04:08.072150 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:08.072035    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 23:04:08.083381 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:08.083365    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 23:04:29.982494 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:29.982460    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"]
Oct 30 23:04:29.982986 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:29.982814    5896 cpu_manager.go:414] "containerMap: removing container" podUID="55eb843f-c309-4127-9bfa-2a8e3e6444fc" containerName="storage-initializer"
Oct 30 23:04:29.982986 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:29.982825    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55eb843f-c309-4127-9bfa-2a8e3e6444fc" containerName="storage-initializer"
Oct 30 23:04:29.982986 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:29.982905    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="55eb843f-c309-4127-9bfa-2a8e3e6444fc" containerName="storage-initializer"
Oct 30 23:04:29.982986 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:29.982917    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="55eb843f-c309-4127-9bfa-2a8e3e6444fc" containerName="storage-initializer"
Oct 30 23:04:29.982986 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:29.982981    5896 cpu_manager.go:414] "containerMap: removing container" podUID="55eb843f-c309-4127-9bfa-2a8e3e6444fc" containerName="storage-initializer"
Oct 30 23:04:29.982986 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:29.982987    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55eb843f-c309-4127-9bfa-2a8e3e6444fc" containerName="storage-initializer"
Oct 30 23:04:29.986146 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:29.986131    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.032928 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.032904    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/46915b16-9347-4641-a0eb-8e41283ef433-proxy-tls\") pod \"isvc-paddle-runtime-predictor-794db68b58-89h6h\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") " pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.033034 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.032930    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/46915b16-9347-4641-a0eb-8e41283ef433-kube-rbac-proxy-sar-config\") pod \"isvc-paddle-runtime-predictor-794db68b58-89h6h\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") " pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.033034 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.032958    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/46915b16-9347-4641-a0eb-8e41283ef433-cabundle-cert\") pod \"isvc-paddle-runtime-predictor-794db68b58-89h6h\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") " pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.033107 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.033049    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-265vw\" (UniqueName: \"kubernetes.io/projected/46915b16-9347-4641-a0eb-8e41283ef433-kube-api-access-265vw\") pod \"isvc-paddle-runtime-predictor-794db68b58-89h6h\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") " pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.033141 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.033106    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/46915b16-9347-4641-a0eb-8e41283ef433-kserve-provision-location\") pod \"isvc-paddle-runtime-predictor-794db68b58-89h6h\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") " pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.041661 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.041613    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"kube-root-ca.crt\""
Oct 30 23:04:30.091356 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.091299    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"openshift-service-ca.crt\""
Oct 30 23:04:30.091356 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.091302    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-paddle-runtime-kube-rbac-proxy-sar-config\""
Oct 30 23:04:30.091356 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.091292    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 23:04:30.091676 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.091653    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-paddle-runtime-predictor-serving-cert\""
Oct 30 23:04:30.091676 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.091667    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"default-dockercfg-nnb5w\""
Oct 30 23:04:30.091676 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.091656    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"seaweedfs-s3-creds\""
Oct 30 23:04:30.093089 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.093062    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"]
Oct 30 23:04:30.134087 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.134057    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-265vw\" (UniqueName: \"kubernetes.io/projected/46915b16-9347-4641-a0eb-8e41283ef433-kube-api-access-265vw\") pod \"isvc-paddle-runtime-predictor-794db68b58-89h6h\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") " pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.134198 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.134099    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/46915b16-9347-4641-a0eb-8e41283ef433-kserve-provision-location\") pod \"isvc-paddle-runtime-predictor-794db68b58-89h6h\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") " pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.134198 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.134154    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/46915b16-9347-4641-a0eb-8e41283ef433-proxy-tls\") pod \"isvc-paddle-runtime-predictor-794db68b58-89h6h\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") " pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.134198 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.134179    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/46915b16-9347-4641-a0eb-8e41283ef433-kube-rbac-proxy-sar-config\") pod \"isvc-paddle-runtime-predictor-794db68b58-89h6h\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") " pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.134358 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.134222    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/46915b16-9347-4641-a0eb-8e41283ef433-cabundle-cert\") pod \"isvc-paddle-runtime-predictor-794db68b58-89h6h\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") " pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.134587 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.134565    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/46915b16-9347-4641-a0eb-8e41283ef433-kserve-provision-location\") pod \"isvc-paddle-runtime-predictor-794db68b58-89h6h\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") " pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.134846 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.134827    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/46915b16-9347-4641-a0eb-8e41283ef433-kube-rbac-proxy-sar-config\") pod \"isvc-paddle-runtime-predictor-794db68b58-89h6h\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") " pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.134937 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.134916    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/46915b16-9347-4641-a0eb-8e41283ef433-cabundle-cert\") pod \"isvc-paddle-runtime-predictor-794db68b58-89h6h\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") " pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.136948 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.136930    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/46915b16-9347-4641-a0eb-8e41283ef433-proxy-tls\") pod \"isvc-paddle-runtime-predictor-794db68b58-89h6h\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") " pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.198773 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.198751    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-265vw\" (UniqueName: \"kubernetes.io/projected/46915b16-9347-4641-a0eb-8e41283ef433-kube-api-access-265vw\") pod \"isvc-paddle-runtime-predictor-794db68b58-89h6h\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") " pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.302906 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.302856    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:30.725437 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:30.725364    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"]
Oct 30 23:04:30.727944 ip-10-0-138-202 kubenswrapper[5896]: W1030 23:04:30.727923    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46915b16_9347_4641_a0eb_8e41283ef433.slice/crio-b8083d9c2af6d26fd13db184c7faf6a2d4b6ff672a909e4483ef5bb7fdc49b18 WatchSource:0}: Error finding container b8083d9c2af6d26fd13db184c7faf6a2d4b6ff672a909e4483ef5bb7fdc49b18: Status 404 returned error can't find the container with id b8083d9c2af6d26fd13db184c7faf6a2d4b6ff672a909e4483ef5bb7fdc49b18
Oct 30 23:04:31.485936 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:31.485895    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" event={"ID":"46915b16-9347-4641-a0eb-8e41283ef433","Type":"ContainerStarted","Data":"a8d529a012fdc109367d8516e3e9905b3b7db47ffae985b45fc09e4a23d53478"}
Oct 30 23:04:31.486321 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:31.485967    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" event={"ID":"46915b16-9347-4641-a0eb-8e41283ef433","Type":"ContainerStarted","Data":"b8083d9c2af6d26fd13db184c7faf6a2d4b6ff672a909e4483ef5bb7fdc49b18"}
Oct 30 23:04:32.629579 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:04:32.629545    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:04:36.513190 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:36.513156    5896 generic.go:356] "Generic (PLEG): container finished" podID="46915b16-9347-4641-a0eb-8e41283ef433" containerID="a8d529a012fdc109367d8516e3e9905b3b7db47ffae985b45fc09e4a23d53478" exitCode=0
Oct 30 23:04:36.513682 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:36.513235    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" event={"ID":"46915b16-9347-4641-a0eb-8e41283ef433","Type":"ContainerDied","Data":"a8d529a012fdc109367d8516e3e9905b3b7db47ffae985b45fc09e4a23d53478"}
Oct 30 23:04:36.514662 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:36.514630    5896 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 23:04:49.574179 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:49.574147    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" event={"ID":"46915b16-9347-4641-a0eb-8e41283ef433","Type":"ContainerStarted","Data":"759ed5925b403d4832f84d4ff4985a1f2b0c4fc62ce4eb930364b39f689926fa"}
Oct 30 23:04:49.574558 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:49.574215    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" event={"ID":"46915b16-9347-4641-a0eb-8e41283ef433","Type":"ContainerStarted","Data":"f81d00659e948f9338d17f44c91b8505b1346a22d712a2a38ca75cf1f85957cd"}
Oct 30 23:04:49.574558 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:49.574418    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:49.750978 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:49.750930    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" podStartSLOduration=2.800542927 podStartE2EDuration="20.750917019s" totalImagesPullingTime="12.572713242s" totalInitContainerRuntime="5.37766085s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:04:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:04:49.749360094 +0000 UTC m=+3342.160826062" watchObservedRunningTime="2026-10-30 23:04:49.750917019 +0000 UTC m=+3342.162382986"
Oct 30 23:04:50.578561 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:50.578530    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:50.579740 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:50.579703    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 30 23:04:51.582501 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:51.582457    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 30 23:04:56.587185 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:56.587161    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:04:56.587709 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:04:56.587681    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 30 23:05:02.625215 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:05:02.625175    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:05:04.425223 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.425194    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"]
Oct 30 23:05:04.427653 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.427625    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:04.482519 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.482495    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"kube-root-ca.crt\""
Oct 30 23:05:04.519498 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.519469    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"openshift-service-ca.crt\""
Oct 30 23:05:04.519603 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.519503    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 23:05:04.519603 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.519503    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-lightgbm-v2-kserve-kube-rbac-proxy-sar-config\""
Oct 30 23:05:04.532268 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.532242    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"seaweedfs-s3-creds\""
Oct 30 23:05:04.532372 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.532288    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-lightgbm-v2-kserve-predictor-serving-cert\""
Oct 30 23:05:04.532372 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.532336    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"default-dockercfg-s9cqw\""
Oct 30 23:05:04.533619 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.533602    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"]
Oct 30 23:05:04.623119 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.623079    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnm9p\" (UniqueName: \"kubernetes.io/projected/38819f95-75bc-4996-8856-02fab2bf074b-kube-api-access-nnm9p\") pod \"isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") " pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:04.623293 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.623133    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/38819f95-75bc-4996-8856-02fab2bf074b-kserve-provision-location\") pod \"isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") " pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:04.623293 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.623214    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/38819f95-75bc-4996-8856-02fab2bf074b-proxy-tls\") pod \"isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") " pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:04.623293 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.623270    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/38819f95-75bc-4996-8856-02fab2bf074b-kube-rbac-proxy-sar-config\") pod \"isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") " pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:04.623415 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.623398    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/38819f95-75bc-4996-8856-02fab2bf074b-cabundle-cert\") pod \"isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") " pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:04.724305 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.724266    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/38819f95-75bc-4996-8856-02fab2bf074b-kube-rbac-proxy-sar-config\") pod \"isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") " pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:04.724495 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.724343    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/38819f95-75bc-4996-8856-02fab2bf074b-cabundle-cert\") pod \"isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") " pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:04.724495 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.724367    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnm9p\" (UniqueName: \"kubernetes.io/projected/38819f95-75bc-4996-8856-02fab2bf074b-kube-api-access-nnm9p\") pod \"isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") " pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:04.724495 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.724387    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/38819f95-75bc-4996-8856-02fab2bf074b-kserve-provision-location\") pod \"isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") " pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:04.724495 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.724409    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/38819f95-75bc-4996-8856-02fab2bf074b-proxy-tls\") pod \"isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") " pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:04.724878 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.724851    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/38819f95-75bc-4996-8856-02fab2bf074b-kserve-provision-location\") pod \"isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") " pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:04.725009 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.724992    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/38819f95-75bc-4996-8856-02fab2bf074b-cabundle-cert\") pod \"isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") " pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:04.725103 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.725085    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/38819f95-75bc-4996-8856-02fab2bf074b-kube-rbac-proxy-sar-config\") pod \"isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") " pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:04.727063 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.727040    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/38819f95-75bc-4996-8856-02fab2bf074b-proxy-tls\") pod \"isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") " pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:04.788198 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:04.788167    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnm9p\" (UniqueName: \"kubernetes.io/projected/38819f95-75bc-4996-8856-02fab2bf074b-kube-api-access-nnm9p\") pod \"isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") " pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:05.040907 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:05.040812    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:05.465572 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:05.465541    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"]
Oct 30 23:05:05.468076 ip-10-0-138-202 kubenswrapper[5896]: W1030 23:05:05.468048    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38819f95_75bc_4996_8856_02fab2bf074b.slice/crio-c41a108360d2599cae95a6adf1e5b547b0c7c4ed40cccabbe1a567836df9e9d1 WatchSource:0}: Error finding container c41a108360d2599cae95a6adf1e5b547b0c7c4ed40cccabbe1a567836df9e9d1: Status 404 returned error can't find the container with id c41a108360d2599cae95a6adf1e5b547b0c7c4ed40cccabbe1a567836df9e9d1
Oct 30 23:05:05.642051 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:05.642016    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52" event={"ID":"38819f95-75bc-4996-8856-02fab2bf074b","Type":"ContainerStarted","Data":"571aa5381a25f3c50c76f7637189c89382eee51d2e16e6aa57334cb966a667ea"}
Oct 30 23:05:05.642195 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:05.642083    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52" event={"ID":"38819f95-75bc-4996-8856-02fab2bf074b","Type":"ContainerStarted","Data":"c41a108360d2599cae95a6adf1e5b547b0c7c4ed40cccabbe1a567836df9e9d1"}
Oct 30 23:05:06.588214 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:06.588176    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 30 23:05:09.660173 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:09.660143    5896 generic.go:356] "Generic (PLEG): container finished" podID="38819f95-75bc-4996-8856-02fab2bf074b" containerID="571aa5381a25f3c50c76f7637189c89382eee51d2e16e6aa57334cb966a667ea" exitCode=0
Oct 30 23:05:09.660514 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:09.660204    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52" event={"ID":"38819f95-75bc-4996-8856-02fab2bf074b","Type":"ContainerDied","Data":"571aa5381a25f3c50c76f7637189c89382eee51d2e16e6aa57334cb966a667ea"}
Oct 30 23:05:16.588540 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:16.588493    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 30 23:05:17.703005 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:17.702971    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52" event={"ID":"38819f95-75bc-4996-8856-02fab2bf074b","Type":"ContainerStarted","Data":"2dfbb09c25c8078f55bbcffd55446aa438a612556f2f869458d219474e1042ea"}
Oct 30 23:05:17.703509 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:17.703046    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52" event={"ID":"38819f95-75bc-4996-8856-02fab2bf074b","Type":"ContainerStarted","Data":"a6d15e04aefb4e81b374a60c2876827f0206c6570b1558d930b5bcb1a7915019"}
Oct 30 23:05:17.703509 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:17.703335    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:17.703509 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:17.703392    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:17.704706 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:17.704679    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52" podUID="38819f95-75bc-4996-8856-02fab2bf074b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Oct 30 23:05:17.878201 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:17.878141    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52" podStartSLOduration=2.346645163 podStartE2EDuration="13.87811645s" totalImagesPullingTime="7.543271349s" totalInitContainerRuntime="3.988199938s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:05:17.875782046 +0000 UTC m=+3370.287248013" watchObservedRunningTime="2026-10-30 23:05:17.87811645 +0000 UTC m=+3370.289582418"
Oct 30 23:05:18.707808 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:18.707767    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52" podUID="38819f95-75bc-4996-8856-02fab2bf074b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Oct 30 23:05:23.713017 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:23.712988    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:23.713472 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:23.713428    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52" podUID="38819f95-75bc-4996-8856-02fab2bf074b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Oct 30 23:05:26.587924 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:26.587886    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 30 23:05:32.629152 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:05:32.629114    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:05:33.714715 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:33.714688    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:36.588846 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:36.588814    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:05:44.656132 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:44.656099    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"]
Oct 30 23:05:44.656556 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:44.656346    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52" podUID="38819f95-75bc-4996-8856-02fab2bf074b" containerName="kserve-container" containerID="cri-o://a6d15e04aefb4e81b374a60c2876827f0206c6570b1558d930b5bcb1a7915019" gracePeriod=30
Oct 30 23:05:44.656556 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:44.656414    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52" podUID="38819f95-75bc-4996-8856-02fab2bf074b" containerName="kube-rbac-proxy" containerID="cri-o://2dfbb09c25c8078f55bbcffd55446aa438a612556f2f869458d219474e1042ea" gracePeriod=30
Oct 30 23:05:44.823134 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:44.823104    5896 generic.go:356] "Generic (PLEG): container finished" podID="38819f95-75bc-4996-8856-02fab2bf074b" containerID="2dfbb09c25c8078f55bbcffd55446aa438a612556f2f869458d219474e1042ea" exitCode=0
Oct 30 23:05:44.823289 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:44.823183    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52" event={"ID":"38819f95-75bc-4996-8856-02fab2bf074b","Type":"ContainerDied","Data":"2dfbb09c25c8078f55bbcffd55446aa438a612556f2f869458d219474e1042ea"}
Oct 30 23:05:45.396310 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.396289    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:45.469749 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.469724    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38819f95-75bc-4996-8856-02fab2bf074b-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/38819f95-75bc-4996-8856-02fab2bf074b-kube-rbac-proxy-sar-config\") pod \"38819f95-75bc-4996-8856-02fab2bf074b\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") "
Oct 30 23:05:45.469854 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.469777    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38819f95-75bc-4996-8856-02fab2bf074b-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/38819f95-75bc-4996-8856-02fab2bf074b-kserve-provision-location\") pod \"38819f95-75bc-4996-8856-02fab2bf074b\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") "
Oct 30 23:05:45.469854 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.469831    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38819f95-75bc-4996-8856-02fab2bf074b-kube-api-access-nnm9p\" (UniqueName: \"kubernetes.io/projected/38819f95-75bc-4996-8856-02fab2bf074b-kube-api-access-nnm9p\") pod \"38819f95-75bc-4996-8856-02fab2bf074b\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") "
Oct 30 23:05:45.469941 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.469859    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38819f95-75bc-4996-8856-02fab2bf074b-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/38819f95-75bc-4996-8856-02fab2bf074b-cabundle-cert\") pod \"38819f95-75bc-4996-8856-02fab2bf074b\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") "
Oct 30 23:05:45.469941 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.469901    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/38819f95-75bc-4996-8856-02fab2bf074b-proxy-tls\" (UniqueName: \"kubernetes.io/secret/38819f95-75bc-4996-8856-02fab2bf074b-proxy-tls\") pod \"38819f95-75bc-4996-8856-02fab2bf074b\" (UID: \"38819f95-75bc-4996-8856-02fab2bf074b\") "
Oct 30 23:05:45.470173 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.470147    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38819f95-75bc-4996-8856-02fab2bf074b-kserve-provision-location" pod "38819f95-75bc-4996-8856-02fab2bf074b" (UID: "38819f95-75bc-4996-8856-02fab2bf074b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:05:45.470243 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.470168    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38819f95-75bc-4996-8856-02fab2bf074b-kube-rbac-proxy-sar-config" pod "38819f95-75bc-4996-8856-02fab2bf074b" (UID: "38819f95-75bc-4996-8856-02fab2bf074b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:05:45.470243 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.470213    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38819f95-75bc-4996-8856-02fab2bf074b-cabundle-cert" pod "38819f95-75bc-4996-8856-02fab2bf074b" (UID: "38819f95-75bc-4996-8856-02fab2bf074b"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:05:45.472039 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.472016    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38819f95-75bc-4996-8856-02fab2bf074b-proxy-tls" pod "38819f95-75bc-4996-8856-02fab2bf074b" (UID: "38819f95-75bc-4996-8856-02fab2bf074b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:05:45.472120 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.472100    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38819f95-75bc-4996-8856-02fab2bf074b-kube-api-access-nnm9p" pod "38819f95-75bc-4996-8856-02fab2bf074b" (UID: "38819f95-75bc-4996-8856-02fab2bf074b"). InnerVolumeSpecName "kube-api-access-nnm9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:05:45.570878 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.570852    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnm9p\" (UniqueName: \"kubernetes.io/projected/38819f95-75bc-4996-8856-02fab2bf074b-kube-api-access-nnm9p\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:05:45.570878 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.570874    5896 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/38819f95-75bc-4996-8856-02fab2bf074b-cabundle-cert\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:05:45.571009 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.570906    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/38819f95-75bc-4996-8856-02fab2bf074b-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:05:45.571009 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.570916    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/38819f95-75bc-4996-8856-02fab2bf074b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:05:45.571009 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.570925    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/38819f95-75bc-4996-8856-02fab2bf074b-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:05:45.829396 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.829368    5896 generic.go:356] "Generic (PLEG): container finished" podID="38819f95-75bc-4996-8856-02fab2bf074b" containerID="a6d15e04aefb4e81b374a60c2876827f0206c6570b1558d930b5bcb1a7915019" exitCode=0
Oct 30 23:05:45.829727 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.829442    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52" event={"ID":"38819f95-75bc-4996-8856-02fab2bf074b","Type":"ContainerDied","Data":"a6d15e04aefb4e81b374a60c2876827f0206c6570b1558d930b5bcb1a7915019"}
Oct 30 23:05:45.829727 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.829474    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"
Oct 30 23:05:45.829727 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.829481    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52" event={"ID":"38819f95-75bc-4996-8856-02fab2bf074b","Type":"ContainerDied","Data":"c41a108360d2599cae95a6adf1e5b547b0c7c4ed40cccabbe1a567836df9e9d1"}
Oct 30 23:05:45.829727 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.829506    5896 scope.go:122] "RemoveContainer" containerID="2dfbb09c25c8078f55bbcffd55446aa438a612556f2f869458d219474e1042ea"
Oct 30 23:05:45.838044 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.838025    5896 scope.go:122] "RemoveContainer" containerID="a6d15e04aefb4e81b374a60c2876827f0206c6570b1558d930b5bcb1a7915019"
Oct 30 23:05:45.845155 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.845138    5896 scope.go:122] "RemoveContainer" containerID="571aa5381a25f3c50c76f7637189c89382eee51d2e16e6aa57334cb966a667ea"
Oct 30 23:05:45.852623 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.852606    5896 scope.go:122] "RemoveContainer" containerID="2dfbb09c25c8078f55bbcffd55446aa438a612556f2f869458d219474e1042ea"
Oct 30 23:05:45.852925 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:05:45.852896    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dfbb09c25c8078f55bbcffd55446aa438a612556f2f869458d219474e1042ea\": container with ID starting with 2dfbb09c25c8078f55bbcffd55446aa438a612556f2f869458d219474e1042ea not found: ID does not exist" containerID="2dfbb09c25c8078f55bbcffd55446aa438a612556f2f869458d219474e1042ea"
Oct 30 23:05:45.853004 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.852937    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dfbb09c25c8078f55bbcffd55446aa438a612556f2f869458d219474e1042ea"} err="failed to get container status \"2dfbb09c25c8078f55bbcffd55446aa438a612556f2f869458d219474e1042ea\": rpc error: code = NotFound desc = could not find container \"2dfbb09c25c8078f55bbcffd55446aa438a612556f2f869458d219474e1042ea\": container with ID starting with 2dfbb09c25c8078f55bbcffd55446aa438a612556f2f869458d219474e1042ea not found: ID does not exist"
Oct 30 23:05:45.853004 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.852963    5896 scope.go:122] "RemoveContainer" containerID="a6d15e04aefb4e81b374a60c2876827f0206c6570b1558d930b5bcb1a7915019"
Oct 30 23:05:45.853216 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:05:45.853199    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6d15e04aefb4e81b374a60c2876827f0206c6570b1558d930b5bcb1a7915019\": container with ID starting with a6d15e04aefb4e81b374a60c2876827f0206c6570b1558d930b5bcb1a7915019 not found: ID does not exist" containerID="a6d15e04aefb4e81b374a60c2876827f0206c6570b1558d930b5bcb1a7915019"
Oct 30 23:05:45.853254 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.853222    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6d15e04aefb4e81b374a60c2876827f0206c6570b1558d930b5bcb1a7915019"} err="failed to get container status \"a6d15e04aefb4e81b374a60c2876827f0206c6570b1558d930b5bcb1a7915019\": rpc error: code = NotFound desc = could not find container \"a6d15e04aefb4e81b374a60c2876827f0206c6570b1558d930b5bcb1a7915019\": container with ID starting with a6d15e04aefb4e81b374a60c2876827f0206c6570b1558d930b5bcb1a7915019 not found: ID does not exist"
Oct 30 23:05:45.853254 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.853238    5896 scope.go:122] "RemoveContainer" containerID="571aa5381a25f3c50c76f7637189c89382eee51d2e16e6aa57334cb966a667ea"
Oct 30 23:05:45.853446 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:05:45.853432    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"571aa5381a25f3c50c76f7637189c89382eee51d2e16e6aa57334cb966a667ea\": container with ID starting with 571aa5381a25f3c50c76f7637189c89382eee51d2e16e6aa57334cb966a667ea not found: ID does not exist" containerID="571aa5381a25f3c50c76f7637189c89382eee51d2e16e6aa57334cb966a667ea"
Oct 30 23:05:45.853492 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:45.853449    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"571aa5381a25f3c50c76f7637189c89382eee51d2e16e6aa57334cb966a667ea"} err="failed to get container status \"571aa5381a25f3c50c76f7637189c89382eee51d2e16e6aa57334cb966a667ea\": rpc error: code = NotFound desc = could not find container \"571aa5381a25f3c50c76f7637189c89382eee51d2e16e6aa57334cb966a667ea\": container with ID starting with 571aa5381a25f3c50c76f7637189c89382eee51d2e16e6aa57334cb966a667ea not found: ID does not exist"
Oct 30 23:05:46.000687 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:46.000623    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"]
Oct 30 23:05:46.050861 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:46.050837    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-959ee2f135/isvc-lightgbm-v2-kserve-predictor-7679b897cf-6rp52"]
Oct 30 23:05:46.106526 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:46.106501    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"]
Oct 30 23:05:46.106804 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:46.106781    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="kserve-container" containerID="cri-o://f81d00659e948f9338d17f44c91b8505b1346a22d712a2a38ca75cf1f85957cd" gracePeriod=30
Oct 30 23:05:46.106891 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:46.106874    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="kube-rbac-proxy" containerID="cri-o://759ed5925b403d4832f84d4ff4985a1f2b0c4fc62ce4eb930364b39f689926fa" gracePeriod=30
Oct 30 23:05:46.582783 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:46.582742    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.40:8643/healthz\": dial tcp 10.134.0.40:8643: connect: connection refused"
Oct 30 23:05:46.588007 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:46.587979    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 30 23:05:46.837176 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:46.837095    5896 generic.go:356] "Generic (PLEG): container finished" podID="46915b16-9347-4641-a0eb-8e41283ef433" containerID="759ed5925b403d4832f84d4ff4985a1f2b0c4fc62ce4eb930364b39f689926fa" exitCode=0
Oct 30 23:05:46.837539 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:46.837169    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" event={"ID":"46915b16-9347-4641-a0eb-8e41283ef433","Type":"ContainerDied","Data":"759ed5925b403d4832f84d4ff4985a1f2b0c4fc62ce4eb930364b39f689926fa"}
Oct 30 23:05:47.736134 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:47.736098    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38819f95-75bc-4996-8856-02fab2bf074b" path="/var/lib/kubelet/pods/38819f95-75bc-4996-8856-02fab2bf074b/volumes"
Oct 30 23:05:48.847698 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:48.847667    5896 generic.go:356] "Generic (PLEG): container finished" podID="46915b16-9347-4641-a0eb-8e41283ef433" containerID="f81d00659e948f9338d17f44c91b8505b1346a22d712a2a38ca75cf1f85957cd" exitCode=0
Oct 30 23:05:48.848026 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:48.847675    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" event={"ID":"46915b16-9347-4641-a0eb-8e41283ef433","Type":"ContainerDied","Data":"f81d00659e948f9338d17f44c91b8505b1346a22d712a2a38ca75cf1f85957cd"}
Oct 30 23:05:49.377562 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.377541    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:05:49.504506 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.504445    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46915b16-9347-4641-a0eb-8e41283ef433-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/46915b16-9347-4641-a0eb-8e41283ef433-cabundle-cert\") pod \"46915b16-9347-4641-a0eb-8e41283ef433\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") "
Oct 30 23:05:49.504753 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.504516    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46915b16-9347-4641-a0eb-8e41283ef433-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/46915b16-9347-4641-a0eb-8e41283ef433-kserve-provision-location\") pod \"46915b16-9347-4641-a0eb-8e41283ef433\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") "
Oct 30 23:05:49.504753 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.504552    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46915b16-9347-4641-a0eb-8e41283ef433-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/46915b16-9347-4641-a0eb-8e41283ef433-kube-rbac-proxy-sar-config\") pod \"46915b16-9347-4641-a0eb-8e41283ef433\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") "
Oct 30 23:05:49.504753 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.504609    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46915b16-9347-4641-a0eb-8e41283ef433-kube-api-access-265vw\" (UniqueName: \"kubernetes.io/projected/46915b16-9347-4641-a0eb-8e41283ef433-kube-api-access-265vw\") pod \"46915b16-9347-4641-a0eb-8e41283ef433\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") "
Oct 30 23:05:49.504753 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.504685    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46915b16-9347-4641-a0eb-8e41283ef433-proxy-tls\" (UniqueName: \"kubernetes.io/secret/46915b16-9347-4641-a0eb-8e41283ef433-proxy-tls\") pod \"46915b16-9347-4641-a0eb-8e41283ef433\" (UID: \"46915b16-9347-4641-a0eb-8e41283ef433\") "
Oct 30 23:05:49.505003 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.504979    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46915b16-9347-4641-a0eb-8e41283ef433-cabundle-cert" pod "46915b16-9347-4641-a0eb-8e41283ef433" (UID: "46915b16-9347-4641-a0eb-8e41283ef433"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:05:49.505071 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.505007    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46915b16-9347-4641-a0eb-8e41283ef433-kube-rbac-proxy-sar-config" pod "46915b16-9347-4641-a0eb-8e41283ef433" (UID: "46915b16-9347-4641-a0eb-8e41283ef433"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:05:49.507293 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.507245    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46915b16-9347-4641-a0eb-8e41283ef433-kube-api-access-265vw" pod "46915b16-9347-4641-a0eb-8e41283ef433" (UID: "46915b16-9347-4641-a0eb-8e41283ef433"). InnerVolumeSpecName "kube-api-access-265vw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:05:49.508584 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.508562    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46915b16-9347-4641-a0eb-8e41283ef433-proxy-tls" pod "46915b16-9347-4641-a0eb-8e41283ef433" (UID: "46915b16-9347-4641-a0eb-8e41283ef433"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:05:49.516173 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.516149    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46915b16-9347-4641-a0eb-8e41283ef433-kserve-provision-location" pod "46915b16-9347-4641-a0eb-8e41283ef433" (UID: "46915b16-9347-4641-a0eb-8e41283ef433"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:05:49.605822 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.605796    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-265vw\" (UniqueName: \"kubernetes.io/projected/46915b16-9347-4641-a0eb-8e41283ef433-kube-api-access-265vw\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:05:49.605822 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.605820    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/46915b16-9347-4641-a0eb-8e41283ef433-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:05:49.605977 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.605830    5896 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/46915b16-9347-4641-a0eb-8e41283ef433-cabundle-cert\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:05:49.605977 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.605840    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/46915b16-9347-4641-a0eb-8e41283ef433-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:05:49.605977 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.605849    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/46915b16-9347-4641-a0eb-8e41283ef433-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:05:49.853329 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.853250    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h" event={"ID":"46915b16-9347-4641-a0eb-8e41283ef433","Type":"ContainerDied","Data":"b8083d9c2af6d26fd13db184c7faf6a2d4b6ff672a909e4483ef5bb7fdc49b18"}
Oct 30 23:05:49.853329 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.853304    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"
Oct 30 23:05:49.853329 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.853326    5896 scope.go:122] "RemoveContainer" containerID="759ed5925b403d4832f84d4ff4985a1f2b0c4fc62ce4eb930364b39f689926fa"
Oct 30 23:05:49.861708 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.861686    5896 scope.go:122] "RemoveContainer" containerID="f81d00659e948f9338d17f44c91b8505b1346a22d712a2a38ca75cf1f85957cd"
Oct 30 23:05:49.869055 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:49.869039    5896 scope.go:122] "RemoveContainer" containerID="a8d529a012fdc109367d8516e3e9905b3b7db47ffae985b45fc09e4a23d53478"
Oct 30 23:05:50.028443 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:50.028416    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"]
Oct 30 23:05:50.078998 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:50.078976    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-bc5112d610/isvc-paddle-runtime-predictor-794db68b58-89h6h"]
Oct 30 23:05:51.735573 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:51.735539    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46915b16-9347-4641-a0eb-8e41283ef433" path="/var/lib/kubelet/pods/46915b16-9347-4641-a0eb-8e41283ef433/volumes"
Oct 30 23:05:54.494762 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.494727    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"]
Oct 30 23:05:54.495137 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.495033    5896 cpu_manager.go:414] "containerMap: removing container" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="kube-rbac-proxy"
Oct 30 23:05:54.495137 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.495043    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="kube-rbac-proxy"
Oct 30 23:05:54.495137 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.495058    5896 cpu_manager.go:414] "containerMap: removing container" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="storage-initializer"
Oct 30 23:05:54.495137 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.495062    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="storage-initializer"
Oct 30 23:05:54.495137 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.495075    5896 cpu_manager.go:414] "containerMap: removing container" podUID="38819f95-75bc-4996-8856-02fab2bf074b" containerName="storage-initializer"
Oct 30 23:05:54.495137 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.495080    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38819f95-75bc-4996-8856-02fab2bf074b" containerName="storage-initializer"
Oct 30 23:05:54.495137 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.495088    5896 cpu_manager.go:414] "containerMap: removing container" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="kserve-container"
Oct 30 23:05:54.495137 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.495092    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="kserve-container"
Oct 30 23:05:54.495137 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.495099    5896 cpu_manager.go:414] "containerMap: removing container" podUID="38819f95-75bc-4996-8856-02fab2bf074b" containerName="kube-rbac-proxy"
Oct 30 23:05:54.495137 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.495104    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38819f95-75bc-4996-8856-02fab2bf074b" containerName="kube-rbac-proxy"
Oct 30 23:05:54.495137 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.495122    5896 cpu_manager.go:414] "containerMap: removing container" podUID="38819f95-75bc-4996-8856-02fab2bf074b" containerName="kserve-container"
Oct 30 23:05:54.495137 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.495126    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38819f95-75bc-4996-8856-02fab2bf074b" containerName="kserve-container"
Oct 30 23:05:54.495493 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.495167    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="kserve-container"
Oct 30 23:05:54.495493 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.495175    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="38819f95-75bc-4996-8856-02fab2bf074b" containerName="kube-rbac-proxy"
Oct 30 23:05:54.495493 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.495183    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="38819f95-75bc-4996-8856-02fab2bf074b" containerName="kserve-container"
Oct 30 23:05:54.495493 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.495190    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="46915b16-9347-4641-a0eb-8e41283ef433" containerName="kube-rbac-proxy"
Oct 30 23:05:54.500709 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.500692    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:54.555833 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.555809    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 23:05:54.555949 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.555858    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"seaweedfs-s3-creds\""
Oct 30 23:05:54.605449 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.605420    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"openshift-service-ca.crt\""
Oct 30 23:05:54.605600 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.605446    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"kube-root-ca.crt\""
Oct 30 23:05:54.605600 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.605425    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-paddle-v2-kserve-kube-rbac-proxy-sar-config\""
Oct 30 23:05:54.605600 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.605511    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-paddle-v2-kserve-predictor-serving-cert\""
Oct 30 23:05:54.605600 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.605571    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"default-dockercfg-nnb5w\""
Oct 30 23:05:54.607250 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.607233    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"]
Oct 30 23:05:54.643539 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.643509    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e68ee25-883c-41c4-9093-2f81fdf565de-proxy-tls\") pod \"isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") " pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:54.643692 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.643588    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8e68ee25-883c-41c4-9093-2f81fdf565de-cabundle-cert\") pod \"isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") " pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:54.643692 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.643626    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e68ee25-883c-41c4-9093-2f81fdf565de-kube-rbac-proxy-sar-config\") pod \"isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") " pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:54.643831 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.643729    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hr95\" (UniqueName: \"kubernetes.io/projected/8e68ee25-883c-41c4-9093-2f81fdf565de-kube-api-access-6hr95\") pod \"isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") " pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:54.643831 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.643765    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8e68ee25-883c-41c4-9093-2f81fdf565de-kserve-provision-location\") pod \"isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") " pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:54.744836 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.744782    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e68ee25-883c-41c4-9093-2f81fdf565de-proxy-tls\") pod \"isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") " pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:54.744836 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.744816    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8e68ee25-883c-41c4-9093-2f81fdf565de-cabundle-cert\") pod \"isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") " pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:54.744836 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.744835    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e68ee25-883c-41c4-9093-2f81fdf565de-kube-rbac-proxy-sar-config\") pod \"isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") " pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:54.745032 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.744870    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hr95\" (UniqueName: \"kubernetes.io/projected/8e68ee25-883c-41c4-9093-2f81fdf565de-kube-api-access-6hr95\") pod \"isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") " pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:54.745032 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.744904    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8e68ee25-883c-41c4-9093-2f81fdf565de-kserve-provision-location\") pod \"isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") " pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:54.745324 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.745304    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8e68ee25-883c-41c4-9093-2f81fdf565de-kserve-provision-location\") pod \"isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") " pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:54.745573 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.745557    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8e68ee25-883c-41c4-9093-2f81fdf565de-cabundle-cert\") pod \"isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") " pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:54.745718 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.745692    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e68ee25-883c-41c4-9093-2f81fdf565de-kube-rbac-proxy-sar-config\") pod \"isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") " pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:54.747453 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.747431    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e68ee25-883c-41c4-9093-2f81fdf565de-proxy-tls\") pod \"isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") " pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:54.808364 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.808336    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hr95\" (UniqueName: \"kubernetes.io/projected/8e68ee25-883c-41c4-9093-2f81fdf565de-kube-api-access-6hr95\") pod \"isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") " pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:54.813153 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:54.813134    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:05:55.236175 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:55.236150    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"]
Oct 30 23:05:55.238745 ip-10-0-138-202 kubenswrapper[5896]: W1030 23:05:55.238715    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e68ee25_883c_41c4_9093_2f81fdf565de.slice/crio-55f6e13180831c8a6594124d0addcae983c1ab65910a2cca9d9611d0a9c0422a WatchSource:0}: Error finding container 55f6e13180831c8a6594124d0addcae983c1ab65910a2cca9d9611d0a9c0422a: Status 404 returned error can't find the container with id 55f6e13180831c8a6594124d0addcae983c1ab65910a2cca9d9611d0a9c0422a
Oct 30 23:05:55.884697 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:55.884664    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" event={"ID":"8e68ee25-883c-41c4-9093-2f81fdf565de","Type":"ContainerStarted","Data":"502e938278d86b3da2489f6e270cc41637360cd92982cf6fd529956c1e21d913"}
Oct 30 23:05:55.885102 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:05:55.884721    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" event={"ID":"8e68ee25-883c-41c4-9093-2f81fdf565de","Type":"ContainerStarted","Data":"55f6e13180831c8a6594124d0addcae983c1ab65910a2cca9d9611d0a9c0422a"}
Oct 30 23:06:00.906484 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:00.906449    5896 generic.go:356] "Generic (PLEG): container finished" podID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerID="502e938278d86b3da2489f6e270cc41637360cd92982cf6fd529956c1e21d913" exitCode=0
Oct 30 23:06:00.906484 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:00.906485    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" event={"ID":"8e68ee25-883c-41c4-9093-2f81fdf565de","Type":"ContainerDied","Data":"502e938278d86b3da2489f6e270cc41637360cd92982cf6fd529956c1e21d913"}
Oct 30 23:06:01.912652 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:01.912617    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" event={"ID":"8e68ee25-883c-41c4-9093-2f81fdf565de","Type":"ContainerStarted","Data":"06a83c60e38f1cb6352769682590670cc27b15f853bfe386c76f647b1a97767e"}
Oct 30 23:06:01.913054 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:01.912687    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" event={"ID":"8e68ee25-883c-41c4-9093-2f81fdf565de","Type":"ContainerStarted","Data":"671d7b24c7ad11a40d183f38dd55ca9a981b04c9c2e0e8996e85d0eaaf7c156e"}
Oct 30 23:06:01.913054 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:01.912867    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:06:02.042078 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:02.042019    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" podStartSLOduration=3.223012668 podStartE2EDuration="8.042006513s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.818993845s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:05:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:06:02.041600641 +0000 UTC m=+3414.453066610" watchObservedRunningTime="2026-10-30 23:06:02.042006513 +0000 UTC m=+3414.453472480"
Oct 30 23:06:02.626007 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:06:02.625964    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:06:02.917004 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:02.916927    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:06:02.918157 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:02.918122    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 30 23:06:03.920538 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:03.920495    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 30 23:06:08.924842 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:08.924814    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:06:08.925330 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:08.925304    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 30 23:06:18.925382 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:18.925295    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 30 23:06:28.925572 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:28.925535    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 30 23:06:32.628689 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:06:32.628613    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:06:38.925475 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:38.925438    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 30 23:06:44.782922 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:44.782889    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"]
Oct 30 23:06:44.786442 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:44.786425    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:44.841441 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:44.841421    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"kube-root-ca.crt\""
Oct 30 23:06:44.891519 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:44.891499    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-sklearn-mcp-kube-rbac-proxy-sar-config\""
Oct 30 23:06:44.891681 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:44.891571    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 23:06:44.891764 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:44.891677    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"openshift-service-ca.crt\""
Oct 30 23:06:44.891764 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:44.891688    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"default-dockercfg-s9cqw\""
Oct 30 23:06:44.891764 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:44.891618    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"seaweedfs-s3-creds\""
Oct 30 23:06:44.891764 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:44.891706    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-sklearn-mcp-predictor-serving-cert\""
Oct 30 23:06:44.892823 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:44.892805    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"]
Oct 30 23:06:44.930911 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:44.930887    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw86w\" (UniqueName: \"kubernetes.io/projected/c846887e-0c76-4b05-8404-05903f2dd163-kube-api-access-rw86w\") pod \"isvc-sklearn-mcp-predictor-579ff6755f-xvm66\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") " pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:44.931006 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:44.930923    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c846887e-0c76-4b05-8404-05903f2dd163-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-mcp-predictor-579ff6755f-xvm66\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") " pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:44.931006 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:44.930951    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c846887e-0c76-4b05-8404-05903f2dd163-kserve-provision-location\") pod \"isvc-sklearn-mcp-predictor-579ff6755f-xvm66\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") " pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:44.931125 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:44.931096    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c846887e-0c76-4b05-8404-05903f2dd163-cabundle-cert\") pod \"isvc-sklearn-mcp-predictor-579ff6755f-xvm66\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") " pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:44.931201 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:44.931188    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c846887e-0c76-4b05-8404-05903f2dd163-proxy-tls\") pod \"isvc-sklearn-mcp-predictor-579ff6755f-xvm66\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") " pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:45.032092 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:45.032065    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw86w\" (UniqueName: \"kubernetes.io/projected/c846887e-0c76-4b05-8404-05903f2dd163-kube-api-access-rw86w\") pod \"isvc-sklearn-mcp-predictor-579ff6755f-xvm66\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") " pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:45.032198 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:45.032102    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c846887e-0c76-4b05-8404-05903f2dd163-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-mcp-predictor-579ff6755f-xvm66\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") " pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:45.032198 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:45.032143    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c846887e-0c76-4b05-8404-05903f2dd163-kserve-provision-location\") pod \"isvc-sklearn-mcp-predictor-579ff6755f-xvm66\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") " pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:45.032322 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:45.032195    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c846887e-0c76-4b05-8404-05903f2dd163-cabundle-cert\") pod \"isvc-sklearn-mcp-predictor-579ff6755f-xvm66\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") " pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:45.032322 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:45.032251    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c846887e-0c76-4b05-8404-05903f2dd163-proxy-tls\") pod \"isvc-sklearn-mcp-predictor-579ff6755f-xvm66\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") " pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:45.032585 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:45.032558    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c846887e-0c76-4b05-8404-05903f2dd163-kserve-provision-location\") pod \"isvc-sklearn-mcp-predictor-579ff6755f-xvm66\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") " pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:45.032850 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:45.032832    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c846887e-0c76-4b05-8404-05903f2dd163-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-mcp-predictor-579ff6755f-xvm66\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") " pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:45.032903 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:45.032887    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c846887e-0c76-4b05-8404-05903f2dd163-cabundle-cert\") pod \"isvc-sklearn-mcp-predictor-579ff6755f-xvm66\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") " pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:45.034788 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:45.034742    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c846887e-0c76-4b05-8404-05903f2dd163-proxy-tls\") pod \"isvc-sklearn-mcp-predictor-579ff6755f-xvm66\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") " pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:45.096197 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:45.096173    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw86w\" (UniqueName: \"kubernetes.io/projected/c846887e-0c76-4b05-8404-05903f2dd163-kube-api-access-rw86w\") pod \"isvc-sklearn-mcp-predictor-579ff6755f-xvm66\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") " pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:45.104946 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:45.104928    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:45.527411 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:45.527382    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"]
Oct 30 23:06:45.530157 ip-10-0-138-202 kubenswrapper[5896]: W1030 23:06:45.530126    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc846887e_0c76_4b05_8404_05903f2dd163.slice/crio-23542978bc8cba0569f493a77fab36ca5e9b1934c56838e17e49fa0217cca3f3 WatchSource:0}: Error finding container 23542978bc8cba0569f493a77fab36ca5e9b1934c56838e17e49fa0217cca3f3: Status 404 returned error can't find the container with id 23542978bc8cba0569f493a77fab36ca5e9b1934c56838e17e49fa0217cca3f3
Oct 30 23:06:46.091774 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:46.091738    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" event={"ID":"c846887e-0c76-4b05-8404-05903f2dd163","Type":"ContainerStarted","Data":"de223ede8066e551a8e0f0dc7eddcda58de5c0993b3548ceceee7196345baec9"}
Oct 30 23:06:46.092254 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:46.091818    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" event={"ID":"c846887e-0c76-4b05-8404-05903f2dd163","Type":"ContainerStarted","Data":"23542978bc8cba0569f493a77fab36ca5e9b1934c56838e17e49fa0217cca3f3"}
Oct 30 23:06:48.926665 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:48.926616    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:06:50.114834 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:50.114801    5896 generic.go:356] "Generic (PLEG): container finished" podID="c846887e-0c76-4b05-8404-05903f2dd163" containerID="de223ede8066e551a8e0f0dc7eddcda58de5c0993b3548ceceee7196345baec9" exitCode=0
Oct 30 23:06:50.115293 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:50.114862    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" event={"ID":"c846887e-0c76-4b05-8404-05903f2dd163","Type":"ContainerDied","Data":"de223ede8066e551a8e0f0dc7eddcda58de5c0993b3548ceceee7196345baec9"}
Oct 30 23:06:51.121068 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:51.121029    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" event={"ID":"c846887e-0c76-4b05-8404-05903f2dd163","Type":"ContainerStarted","Data":"3a1fb098837edf1b48fed1ca1326443fc359074d12fe04b937e0bdcd6eb5c197"}
Oct 30 23:06:54.136183 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:54.136153    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" event={"ID":"c846887e-0c76-4b05-8404-05903f2dd163","Type":"ContainerStarted","Data":"723196d2fc85e1ac501b0061141667b5a4fd3d1806f96a21531659176c0238b6"}
Oct 30 23:06:54.136578 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:54.136217    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:54.136578 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:54.136240    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" event={"ID":"c846887e-0c76-4b05-8404-05903f2dd163","Type":"ContainerStarted","Data":"6785771587d203a64ce3ab4916d7aeb009fc7bcc166c1c21e083ded6300fdea8"}
Oct 30 23:06:54.136578 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:54.136265    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:54.292390 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:54.292339    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" podStartSLOduration=2.555525503 podStartE2EDuration="10.292323472s" totalImagesPullingTime="3.713189756s" totalInitContainerRuntime="4.023608213s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:06:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:06:54.266095652 +0000 UTC m=+3466.677561647" watchObservedRunningTime="2026-10-30 23:06:54.292323472 +0000 UTC m=+3466.703789440"
Oct 30 23:06:55.140379 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:55.140343    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:06:55.308419 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:55.308386    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"]
Oct 30 23:06:55.308629 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:55.308597    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerName="kserve-container" containerID="cri-o://671d7b24c7ad11a40d183f38dd55ca9a981b04c9c2e0e8996e85d0eaaf7c156e" gracePeriod=30
Oct 30 23:06:55.308785 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:55.308675    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerName="kube-rbac-proxy" containerID="cri-o://06a83c60e38f1cb6352769682590670cc27b15f853bfe386c76f647b1a97767e" gracePeriod=30
Oct 30 23:06:56.146820 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:56.146789    5896 generic.go:356] "Generic (PLEG): container finished" podID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerID="06a83c60e38f1cb6352769682590670cc27b15f853bfe386c76f647b1a97767e" exitCode=0
Oct 30 23:06:56.147215 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:56.146860    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" event={"ID":"8e68ee25-883c-41c4-9093-2f81fdf565de","Type":"ContainerDied","Data":"06a83c60e38f1cb6352769682590670cc27b15f853bfe386c76f647b1a97767e"}
Oct 30 23:06:57.954551 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:57.954522    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:06:58.148789 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.148713    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e68ee25-883c-41c4-9093-2f81fdf565de-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8e68ee25-883c-41c4-9093-2f81fdf565de-cabundle-cert\") pod \"8e68ee25-883c-41c4-9093-2f81fdf565de\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") "
Oct 30 23:06:58.148789 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.148753    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e68ee25-883c-41c4-9093-2f81fdf565de-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8e68ee25-883c-41c4-9093-2f81fdf565de-kserve-provision-location\") pod \"8e68ee25-883c-41c4-9093-2f81fdf565de\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") "
Oct 30 23:06:58.149069 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.148792    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e68ee25-883c-41c4-9093-2f81fdf565de-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e68ee25-883c-41c4-9093-2f81fdf565de-kube-rbac-proxy-sar-config\") pod \"8e68ee25-883c-41c4-9093-2f81fdf565de\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") "
Oct 30 23:06:58.149069 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.148834    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e68ee25-883c-41c4-9093-2f81fdf565de-kube-api-access-6hr95\" (UniqueName: \"kubernetes.io/projected/8e68ee25-883c-41c4-9093-2f81fdf565de-kube-api-access-6hr95\") pod \"8e68ee25-883c-41c4-9093-2f81fdf565de\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") "
Oct 30 23:06:58.149069 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.148855    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8e68ee25-883c-41c4-9093-2f81fdf565de-proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e68ee25-883c-41c4-9093-2f81fdf565de-proxy-tls\") pod \"8e68ee25-883c-41c4-9093-2f81fdf565de\" (UID: \"8e68ee25-883c-41c4-9093-2f81fdf565de\") "
Oct 30 23:06:58.149235 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.149135    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e68ee25-883c-41c4-9093-2f81fdf565de-cabundle-cert" pod "8e68ee25-883c-41c4-9093-2f81fdf565de" (UID: "8e68ee25-883c-41c4-9093-2f81fdf565de"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:06:58.149285 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.149224    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e68ee25-883c-41c4-9093-2f81fdf565de-kube-rbac-proxy-sar-config" pod "8e68ee25-883c-41c4-9093-2f81fdf565de" (UID: "8e68ee25-883c-41c4-9093-2f81fdf565de"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:06:58.151037 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.151017    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e68ee25-883c-41c4-9093-2f81fdf565de-proxy-tls" pod "8e68ee25-883c-41c4-9093-2f81fdf565de" (UID: "8e68ee25-883c-41c4-9093-2f81fdf565de"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:06:58.151097 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.151064    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e68ee25-883c-41c4-9093-2f81fdf565de-kube-api-access-6hr95" pod "8e68ee25-883c-41c4-9093-2f81fdf565de" (UID: "8e68ee25-883c-41c4-9093-2f81fdf565de"). InnerVolumeSpecName "kube-api-access-6hr95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:06:58.157662 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.157611    5896 generic.go:356] "Generic (PLEG): container finished" podID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerID="671d7b24c7ad11a40d183f38dd55ca9a981b04c9c2e0e8996e85d0eaaf7c156e" exitCode=0
Oct 30 23:06:58.157752 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.157681    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" event={"ID":"8e68ee25-883c-41c4-9093-2f81fdf565de","Type":"ContainerDied","Data":"671d7b24c7ad11a40d183f38dd55ca9a981b04c9c2e0e8996e85d0eaaf7c156e"}
Oct 30 23:06:58.157752 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.157716    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"
Oct 30 23:06:58.157851 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.157760    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp" event={"ID":"8e68ee25-883c-41c4-9093-2f81fdf565de","Type":"ContainerDied","Data":"55f6e13180831c8a6594124d0addcae983c1ab65910a2cca9d9611d0a9c0422a"}
Oct 30 23:06:58.157851 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.157786    5896 scope.go:122] "RemoveContainer" containerID="06a83c60e38f1cb6352769682590670cc27b15f853bfe386c76f647b1a97767e"
Oct 30 23:06:58.162749 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.162721    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e68ee25-883c-41c4-9093-2f81fdf565de-kserve-provision-location" pod "8e68ee25-883c-41c4-9093-2f81fdf565de" (UID: "8e68ee25-883c-41c4-9093-2f81fdf565de"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:06:58.169835 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.169820    5896 scope.go:122] "RemoveContainer" containerID="671d7b24c7ad11a40d183f38dd55ca9a981b04c9c2e0e8996e85d0eaaf7c156e"
Oct 30 23:06:58.178818 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.178800    5896 scope.go:122] "RemoveContainer" containerID="502e938278d86b3da2489f6e270cc41637360cd92982cf6fd529956c1e21d913"
Oct 30 23:06:58.186919 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.186902    5896 scope.go:122] "RemoveContainer" containerID="06a83c60e38f1cb6352769682590670cc27b15f853bfe386c76f647b1a97767e"
Oct 30 23:06:58.187184 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:06:58.187165    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06a83c60e38f1cb6352769682590670cc27b15f853bfe386c76f647b1a97767e\": container with ID starting with 06a83c60e38f1cb6352769682590670cc27b15f853bfe386c76f647b1a97767e not found: ID does not exist" containerID="06a83c60e38f1cb6352769682590670cc27b15f853bfe386c76f647b1a97767e"
Oct 30 23:06:58.187241 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.187193    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06a83c60e38f1cb6352769682590670cc27b15f853bfe386c76f647b1a97767e"} err="failed to get container status \"06a83c60e38f1cb6352769682590670cc27b15f853bfe386c76f647b1a97767e\": rpc error: code = NotFound desc = could not find container \"06a83c60e38f1cb6352769682590670cc27b15f853bfe386c76f647b1a97767e\": container with ID starting with 06a83c60e38f1cb6352769682590670cc27b15f853bfe386c76f647b1a97767e not found: ID does not exist"
Oct 30 23:06:58.187241 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.187211    5896 scope.go:122] "RemoveContainer" containerID="671d7b24c7ad11a40d183f38dd55ca9a981b04c9c2e0e8996e85d0eaaf7c156e"
Oct 30 23:06:58.187437 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:06:58.187420    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"671d7b24c7ad11a40d183f38dd55ca9a981b04c9c2e0e8996e85d0eaaf7c156e\": container with ID starting with 671d7b24c7ad11a40d183f38dd55ca9a981b04c9c2e0e8996e85d0eaaf7c156e not found: ID does not exist" containerID="671d7b24c7ad11a40d183f38dd55ca9a981b04c9c2e0e8996e85d0eaaf7c156e"
Oct 30 23:06:58.187499 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.187447    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"671d7b24c7ad11a40d183f38dd55ca9a981b04c9c2e0e8996e85d0eaaf7c156e"} err="failed to get container status \"671d7b24c7ad11a40d183f38dd55ca9a981b04c9c2e0e8996e85d0eaaf7c156e\": rpc error: code = NotFound desc = could not find container \"671d7b24c7ad11a40d183f38dd55ca9a981b04c9c2e0e8996e85d0eaaf7c156e\": container with ID starting with 671d7b24c7ad11a40d183f38dd55ca9a981b04c9c2e0e8996e85d0eaaf7c156e not found: ID does not exist"
Oct 30 23:06:58.187499 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.187468    5896 scope.go:122] "RemoveContainer" containerID="502e938278d86b3da2489f6e270cc41637360cd92982cf6fd529956c1e21d913"
Oct 30 23:06:58.187760 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:06:58.187741    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"502e938278d86b3da2489f6e270cc41637360cd92982cf6fd529956c1e21d913\": container with ID starting with 502e938278d86b3da2489f6e270cc41637360cd92982cf6fd529956c1e21d913 not found: ID does not exist" containerID="502e938278d86b3da2489f6e270cc41637360cd92982cf6fd529956c1e21d913"
Oct 30 23:06:58.187844 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.187767    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"502e938278d86b3da2489f6e270cc41637360cd92982cf6fd529956c1e21d913"} err="failed to get container status \"502e938278d86b3da2489f6e270cc41637360cd92982cf6fd529956c1e21d913\": rpc error: code = NotFound desc = could not find container \"502e938278d86b3da2489f6e270cc41637360cd92982cf6fd529956c1e21d913\": container with ID starting with 502e938278d86b3da2489f6e270cc41637360cd92982cf6fd529956c1e21d913 not found: ID does not exist"
Oct 30 23:06:58.249817 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.249796    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e68ee25-883c-41c4-9093-2f81fdf565de-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:06:58.249910 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.249819    5896 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8e68ee25-883c-41c4-9093-2f81fdf565de-cabundle-cert\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:06:58.249910 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.249833    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8e68ee25-883c-41c4-9093-2f81fdf565de-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:06:58.249910 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.249846    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e68ee25-883c-41c4-9093-2f81fdf565de-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:06:58.249910 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.249859    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hr95\" (UniqueName: \"kubernetes.io/projected/8e68ee25-883c-41c4-9093-2f81fdf565de-kube-api-access-6hr95\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:06:58.584855 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.584826    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"]
Oct 30 23:06:58.635347 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:58.635310    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-bc5112d610/isvc-paddle-v2-kserve-predictor-65664b8fdb-9pxkp"]
Oct 30 23:06:59.735038 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:06:59.734999    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" path="/var/lib/kubelet/pods/8e68ee25-883c-41c4-9093-2f81fdf565de/volumes"
Oct 30 23:07:01.152378 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:01.152354    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:07:02.381890 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.381860    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"]
Oct 30 23:07:02.382258 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.382197    5896 cpu_manager.go:414] "containerMap: removing container" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerName="kube-rbac-proxy"
Oct 30 23:07:02.382258 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.382208    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerName="kube-rbac-proxy"
Oct 30 23:07:02.382258 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.382218    5896 cpu_manager.go:414] "containerMap: removing container" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerName="kserve-container"
Oct 30 23:07:02.382258 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.382223    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerName="kserve-container"
Oct 30 23:07:02.382258 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.382258    5896 cpu_manager.go:414] "containerMap: removing container" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerName="storage-initializer"
Oct 30 23:07:02.382416 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.382263    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerName="storage-initializer"
Oct 30 23:07:02.382416 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.382319    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerName="kube-rbac-proxy"
Oct 30 23:07:02.382416 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.382326    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e68ee25-883c-41c4-9093-2f81fdf565de" containerName="kserve-container"
Oct 30 23:07:02.386528 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.386511    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:02.441438 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.441399    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-pmml-runtime-kube-rbac-proxy-sar-config\""
Oct 30 23:07:02.441576 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.441464    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 23:07:02.441576 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.441531    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"openshift-service-ca.crt\""
Oct 30 23:07:02.441683 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.441633    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-bc5112d610\"/\"kube-root-ca.crt\""
Oct 30 23:07:02.481247 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.481227    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gptrd\" (UniqueName: \"kubernetes.io/projected/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kube-api-access-gptrd\") pod \"isvc-pmml-runtime-predictor-5f474bd489-2wfv6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") " pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:02.481346 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.481271    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/02f326a4-a7ca-4e2e-bf18-f157eade28e6-proxy-tls\") pod \"isvc-pmml-runtime-predictor-5f474bd489-2wfv6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") " pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:02.481346 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.481334    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kserve-provision-location\") pod \"isvc-pmml-runtime-predictor-5f474bd489-2wfv6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") " pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:02.481411 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.481355    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/02f326a4-a7ca-4e2e-bf18-f157eade28e6-cabundle-cert\") pod \"isvc-pmml-runtime-predictor-5f474bd489-2wfv6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") " pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:02.481411 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.481397    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-runtime-predictor-5f474bd489-2wfv6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") " pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:02.491435 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.491385    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"default-dockercfg-nnb5w\""
Oct 30 23:07:02.491435 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.491423    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"seaweedfs-s3-creds\""
Oct 30 23:07:02.491583 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.491448    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-bc5112d610\"/\"isvc-pmml-runtime-predictor-serving-cert\""
Oct 30 23:07:02.492513 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.492493    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"]
Oct 30 23:07:02.582006 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.581983    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-runtime-predictor-5f474bd489-2wfv6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") " pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:02.582108 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.582038    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gptrd\" (UniqueName: \"kubernetes.io/projected/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kube-api-access-gptrd\") pod \"isvc-pmml-runtime-predictor-5f474bd489-2wfv6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") " pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:02.582108 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.582073    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/02f326a4-a7ca-4e2e-bf18-f157eade28e6-proxy-tls\") pod \"isvc-pmml-runtime-predictor-5f474bd489-2wfv6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") " pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:02.582108 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.582099    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kserve-provision-location\") pod \"isvc-pmml-runtime-predictor-5f474bd489-2wfv6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") " pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:02.582259 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.582118    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/02f326a4-a7ca-4e2e-bf18-f157eade28e6-cabundle-cert\") pod \"isvc-pmml-runtime-predictor-5f474bd489-2wfv6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") " pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:02.582504 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.582483    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kserve-provision-location\") pod \"isvc-pmml-runtime-predictor-5f474bd489-2wfv6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") " pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:02.582748 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.582731    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/02f326a4-a7ca-4e2e-bf18-f157eade28e6-cabundle-cert\") pod \"isvc-pmml-runtime-predictor-5f474bd489-2wfv6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") " pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:02.582849 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.582825    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-runtime-predictor-5f474bd489-2wfv6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") " pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:02.584497 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.584481    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/02f326a4-a7ca-4e2e-bf18-f157eade28e6-proxy-tls\") pod \"isvc-pmml-runtime-predictor-5f474bd489-2wfv6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") " pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:02.626103 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:07:02.626076    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:07:02.645894 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.645843    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gptrd\" (UniqueName: \"kubernetes.io/projected/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kube-api-access-gptrd\") pod \"isvc-pmml-runtime-predictor-5f474bd489-2wfv6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") " pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:02.702365 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:02.702339    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:03.124634 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:03.124612    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"]
Oct 30 23:07:03.126874 ip-10-0-138-202 kubenswrapper[5896]: W1030 23:07:03.126846    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02f326a4_a7ca_4e2e_bf18_f157eade28e6.slice/crio-3a2702d80d6f03476f1634d85694eaf817a1ca23ca451252c63952e353006f54 WatchSource:0}: Error finding container 3a2702d80d6f03476f1634d85694eaf817a1ca23ca451252c63952e353006f54: Status 404 returned error can't find the container with id 3a2702d80d6f03476f1634d85694eaf817a1ca23ca451252c63952e353006f54
Oct 30 23:07:03.183282 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:03.183253    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" event={"ID":"02f326a4-a7ca-4e2e-bf18-f157eade28e6","Type":"ContainerStarted","Data":"79c96433ac99839c32d68489821ffb8cbf21b1b46833dff3ea2690d18fa82fe2"}
Oct 30 23:07:03.183472 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:03.183312    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" event={"ID":"02f326a4-a7ca-4e2e-bf18-f157eade28e6","Type":"ContainerStarted","Data":"3a2702d80d6f03476f1634d85694eaf817a1ca23ca451252c63952e353006f54"}
Oct 30 23:07:07.201751 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:07.201717    5896 generic.go:356] "Generic (PLEG): container finished" podID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerID="79c96433ac99839c32d68489821ffb8cbf21b1b46833dff3ea2690d18fa82fe2" exitCode=0
Oct 30 23:07:07.202105 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:07.201802    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" event={"ID":"02f326a4-a7ca-4e2e-bf18-f157eade28e6","Type":"ContainerDied","Data":"79c96433ac99839c32d68489821ffb8cbf21b1b46833dff3ea2690d18fa82fe2"}
Oct 30 23:07:08.208233 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:08.208204    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" event={"ID":"02f326a4-a7ca-4e2e-bf18-f157eade28e6","Type":"ContainerStarted","Data":"c8cb9c46454e53ab2cc1b3d6d17295f7eea26240890deded58e54437b09b088f"}
Oct 30 23:07:08.208656 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:08.208275    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" event={"ID":"02f326a4-a7ca-4e2e-bf18-f157eade28e6","Type":"ContainerStarted","Data":"5e74b54d50a61eee050fdc3391a3c47c158acfb13bf876a0137a89ca5b9227aa"}
Oct 30 23:07:08.208656 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:08.208501    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:08.332442 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:08.332389    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" podStartSLOduration=2.523369909 podStartE2EDuration="6.332373756s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.809003847s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:07:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:07:08.331824947 +0000 UTC m=+3480.743290916" watchObservedRunningTime="2026-10-30 23:07:08.332373756 +0000 UTC m=+3480.743839722"
Oct 30 23:07:09.212795 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:09.212760    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:09.213767 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:09.213742    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Oct 30 23:07:10.216880 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:10.216840    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Oct 30 23:07:15.221022 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:15.220991    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:07:15.221662 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:15.221609    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Oct 30 23:07:21.153836 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:21.153799    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kserve-container" probeResult="failure" output="Get \"http://10.134.0.43:8080/v1/models/isvc-sklearn-mcp\": dial tcp 10.134.0.43:8080: connect: connection refused"
Oct 30 23:07:25.105939 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:25.105896    5896 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kserve-container" probeResult="failure" output="Get \"http://10.134.0.43:8080/v1/models/isvc-sklearn-mcp\": dial tcp 10.134.0.43:8080: connect: connection refused"
Oct 30 23:07:25.222141 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:25.222106    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Oct 30 23:07:31.284741 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:31.284713    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:07:32.628080 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:07:32.628042    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:07:35.222068 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:35.222026    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Oct 30 23:07:45.221572 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:45.221489    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Oct 30 23:07:55.222134 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:07:55.222093    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Oct 30 23:08:01.285585 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:01.285557    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:08:02.625632 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:08:02.625595    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:08:04.643920 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:04.643876    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"]
Oct 30 23:08:04.644328 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:04.644131    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kserve-container" containerID="cri-o://3a1fb098837edf1b48fed1ca1326443fc359074d12fe04b937e0bdcd6eb5c197" gracePeriod=30
Oct 30 23:08:04.644328 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:04.644183    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kserve-agent" containerID="cri-o://6785771587d203a64ce3ab4916d7aeb009fc7bcc166c1c21e083ded6300fdea8" gracePeriod=30
Oct 30 23:08:04.644328 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:04.644193    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kube-rbac-proxy" containerID="cri-o://723196d2fc85e1ac501b0061141667b5a4fd3d1806f96a21531659176c0238b6" gracePeriod=30
Oct 30 23:08:05.221974 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:05.221938    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Oct 30 23:08:05.461846 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:05.461813    5896 generic.go:356] "Generic (PLEG): container finished" podID="c846887e-0c76-4b05-8404-05903f2dd163" containerID="723196d2fc85e1ac501b0061141667b5a4fd3d1806f96a21531659176c0238b6" exitCode=0
Oct 30 23:08:05.462021 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:05.461870    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" event={"ID":"c846887e-0c76-4b05-8404-05903f2dd163","Type":"ContainerDied","Data":"723196d2fc85e1ac501b0061141667b5a4fd3d1806f96a21531659176c0238b6"}
Oct 30 23:08:06.147554 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:06.147509    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.43:8643/healthz\": dial tcp 10.134.0.43:8643: connect: connection refused"
Oct 30 23:08:07.474567 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:07.474537    5896 generic.go:356] "Generic (PLEG): container finished" podID="c846887e-0c76-4b05-8404-05903f2dd163" containerID="3a1fb098837edf1b48fed1ca1326443fc359074d12fe04b937e0bdcd6eb5c197" exitCode=0
Oct 30 23:08:07.474956 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:07.474607    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" event={"ID":"c846887e-0c76-4b05-8404-05903f2dd163","Type":"ContainerDied","Data":"3a1fb098837edf1b48fed1ca1326443fc359074d12fe04b937e0bdcd6eb5c197"}
Oct 30 23:08:11.147903 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:11.147856    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.43:8643/healthz\": dial tcp 10.134.0.43:8643: connect: connection refused"
Oct 30 23:08:11.153266 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:11.153241    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kserve-container" probeResult="failure" output="Get \"http://10.134.0.43:8080/v1/models/isvc-sklearn-mcp\": dial tcp 10.134.0.43:8080: connect: connection refused"
Oct 30 23:08:15.222492 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:15.222447    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Oct 30 23:08:16.147764 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:16.147713    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.43:8643/healthz\": dial tcp 10.134.0.43:8643: connect: connection refused"
Oct 30 23:08:16.147937 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:16.147875    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:08:21.148113 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:21.148072    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.43:8643/healthz\": dial tcp 10.134.0.43:8643: connect: connection refused"
Oct 30 23:08:21.153463 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:21.153436    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kserve-container" probeResult="failure" output="Get \"http://10.134.0.43:8080/v1/models/isvc-sklearn-mcp\": dial tcp 10.134.0.43:8080: connect: connection refused"
Oct 30 23:08:25.221997 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:25.221955    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused"
Oct 30 23:08:26.147586 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:26.147545    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.43:8643/healthz\": dial tcp 10.134.0.43:8643: connect: connection refused"
Oct 30 23:08:31.148367 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:31.148330    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.43:8643/healthz\": dial tcp 10.134.0.43:8643: connect: connection refused"
Oct 30 23:08:31.153770 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:31.153739    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kserve-container" probeResult="failure" output="Get \"http://10.134.0.43:8080/v1/models/isvc-sklearn-mcp\": dial tcp 10.134.0.43:8080: connect: connection refused"
Oct 30 23:08:31.153875 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:31.153865    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:08:32.629941 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:08:32.629902    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:08:33.735485 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:33.735456    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:08:34.835945 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:34.835921    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:08:34.924424 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:34.924364    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c846887e-0c76-4b05-8404-05903f2dd163-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c846887e-0c76-4b05-8404-05903f2dd163-kserve-provision-location\") pod \"c846887e-0c76-4b05-8404-05903f2dd163\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") "
Oct 30 23:08:34.924424 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:34.924403    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c846887e-0c76-4b05-8404-05903f2dd163-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c846887e-0c76-4b05-8404-05903f2dd163-cabundle-cert\") pod \"c846887e-0c76-4b05-8404-05903f2dd163\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") "
Oct 30 23:08:34.924571 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:34.924512    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c846887e-0c76-4b05-8404-05903f2dd163-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c846887e-0c76-4b05-8404-05903f2dd163-kube-rbac-proxy-sar-config\") pod \"c846887e-0c76-4b05-8404-05903f2dd163\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") "
Oct 30 23:08:34.924571 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:34.924544    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c846887e-0c76-4b05-8404-05903f2dd163-kube-api-access-rw86w\" (UniqueName: \"kubernetes.io/projected/c846887e-0c76-4b05-8404-05903f2dd163-kube-api-access-rw86w\") pod \"c846887e-0c76-4b05-8404-05903f2dd163\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") "
Oct 30 23:08:34.924658 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:34.924588    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c846887e-0c76-4b05-8404-05903f2dd163-proxy-tls\" (UniqueName: \"kubernetes.io/secret/c846887e-0c76-4b05-8404-05903f2dd163-proxy-tls\") pod \"c846887e-0c76-4b05-8404-05903f2dd163\" (UID: \"c846887e-0c76-4b05-8404-05903f2dd163\") "
Oct 30 23:08:34.924742 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:34.924721    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c846887e-0c76-4b05-8404-05903f2dd163-kserve-provision-location" pod "c846887e-0c76-4b05-8404-05903f2dd163" (UID: "c846887e-0c76-4b05-8404-05903f2dd163"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:34.924880 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:34.924843    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c846887e-0c76-4b05-8404-05903f2dd163-cabundle-cert" pod "c846887e-0c76-4b05-8404-05903f2dd163" (UID: "c846887e-0c76-4b05-8404-05903f2dd163"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:08:34.924880 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:34.924869    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c846887e-0c76-4b05-8404-05903f2dd163-kube-rbac-proxy-sar-config" pod "c846887e-0c76-4b05-8404-05903f2dd163" (UID: "c846887e-0c76-4b05-8404-05903f2dd163"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:08:34.924880 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:34.924880    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c846887e-0c76-4b05-8404-05903f2dd163-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:08:34.926665 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:34.926626    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c846887e-0c76-4b05-8404-05903f2dd163-proxy-tls" pod "c846887e-0c76-4b05-8404-05903f2dd163" (UID: "c846887e-0c76-4b05-8404-05903f2dd163"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:08:34.926742 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:34.926726    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c846887e-0c76-4b05-8404-05903f2dd163-kube-api-access-rw86w" pod "c846887e-0c76-4b05-8404-05903f2dd163" (UID: "c846887e-0c76-4b05-8404-05903f2dd163"). InnerVolumeSpecName "kube-api-access-rw86w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:08:35.026190 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.026165    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c846887e-0c76-4b05-8404-05903f2dd163-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:08:35.026190 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.026186    5896 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c846887e-0c76-4b05-8404-05903f2dd163-cabundle-cert\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:08:35.026331 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.026196    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c846887e-0c76-4b05-8404-05903f2dd163-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:08:35.026331 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.026207    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rw86w\" (UniqueName: \"kubernetes.io/projected/c846887e-0c76-4b05-8404-05903f2dd163-kube-api-access-rw86w\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:08:35.597404 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.597368    5896 generic.go:356] "Generic (PLEG): container finished" podID="c846887e-0c76-4b05-8404-05903f2dd163" containerID="6785771587d203a64ce3ab4916d7aeb009fc7bcc166c1c21e083ded6300fdea8" exitCode=0
Oct 30 23:08:35.597575 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.597427    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" event={"ID":"c846887e-0c76-4b05-8404-05903f2dd163","Type":"ContainerDied","Data":"6785771587d203a64ce3ab4916d7aeb009fc7bcc166c1c21e083ded6300fdea8"}
Oct 30 23:08:35.597575 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.597481    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66" event={"ID":"c846887e-0c76-4b05-8404-05903f2dd163","Type":"ContainerDied","Data":"23542978bc8cba0569f493a77fab36ca5e9b1934c56838e17e49fa0217cca3f3"}
Oct 30 23:08:35.597575 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.597429    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"
Oct 30 23:08:35.597575 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.597515    5896 scope.go:122] "RemoveContainer" containerID="723196d2fc85e1ac501b0061141667b5a4fd3d1806f96a21531659176c0238b6"
Oct 30 23:08:35.606947 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.606925    5896 scope.go:122] "RemoveContainer" containerID="6785771587d203a64ce3ab4916d7aeb009fc7bcc166c1c21e083ded6300fdea8"
Oct 30 23:08:35.614452 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.614434    5896 scope.go:122] "RemoveContainer" containerID="3a1fb098837edf1b48fed1ca1326443fc359074d12fe04b937e0bdcd6eb5c197"
Oct 30 23:08:35.622201 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.622182    5896 scope.go:122] "RemoveContainer" containerID="de223ede8066e551a8e0f0dc7eddcda58de5c0993b3548ceceee7196345baec9"
Oct 30 23:08:35.629589 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.629569    5896 scope.go:122] "RemoveContainer" containerID="723196d2fc85e1ac501b0061141667b5a4fd3d1806f96a21531659176c0238b6"
Oct 30 23:08:35.629959 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:08:35.629938    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"723196d2fc85e1ac501b0061141667b5a4fd3d1806f96a21531659176c0238b6\": container with ID starting with 723196d2fc85e1ac501b0061141667b5a4fd3d1806f96a21531659176c0238b6 not found: ID does not exist" containerID="723196d2fc85e1ac501b0061141667b5a4fd3d1806f96a21531659176c0238b6"
Oct 30 23:08:35.630000 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.629968    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"723196d2fc85e1ac501b0061141667b5a4fd3d1806f96a21531659176c0238b6"} err="failed to get container status \"723196d2fc85e1ac501b0061141667b5a4fd3d1806f96a21531659176c0238b6\": rpc error: code = NotFound desc = could not find container \"723196d2fc85e1ac501b0061141667b5a4fd3d1806f96a21531659176c0238b6\": container with ID starting with 723196d2fc85e1ac501b0061141667b5a4fd3d1806f96a21531659176c0238b6 not found: ID does not exist"
Oct 30 23:08:35.630000 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.629988    5896 scope.go:122] "RemoveContainer" containerID="6785771587d203a64ce3ab4916d7aeb009fc7bcc166c1c21e083ded6300fdea8"
Oct 30 23:08:35.630226 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:08:35.630209    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6785771587d203a64ce3ab4916d7aeb009fc7bcc166c1c21e083ded6300fdea8\": container with ID starting with 6785771587d203a64ce3ab4916d7aeb009fc7bcc166c1c21e083ded6300fdea8 not found: ID does not exist" containerID="6785771587d203a64ce3ab4916d7aeb009fc7bcc166c1c21e083ded6300fdea8"
Oct 30 23:08:35.630270 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.630233    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6785771587d203a64ce3ab4916d7aeb009fc7bcc166c1c21e083ded6300fdea8"} err="failed to get container status \"6785771587d203a64ce3ab4916d7aeb009fc7bcc166c1c21e083ded6300fdea8\": rpc error: code = NotFound desc = could not find container \"6785771587d203a64ce3ab4916d7aeb009fc7bcc166c1c21e083ded6300fdea8\": container with ID starting with 6785771587d203a64ce3ab4916d7aeb009fc7bcc166c1c21e083ded6300fdea8 not found: ID does not exist"
Oct 30 23:08:35.630270 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.630250    5896 scope.go:122] "RemoveContainer" containerID="3a1fb098837edf1b48fed1ca1326443fc359074d12fe04b937e0bdcd6eb5c197"
Oct 30 23:08:35.630465 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:08:35.630448    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a1fb098837edf1b48fed1ca1326443fc359074d12fe04b937e0bdcd6eb5c197\": container with ID starting with 3a1fb098837edf1b48fed1ca1326443fc359074d12fe04b937e0bdcd6eb5c197 not found: ID does not exist" containerID="3a1fb098837edf1b48fed1ca1326443fc359074d12fe04b937e0bdcd6eb5c197"
Oct 30 23:08:35.630506 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.630470    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a1fb098837edf1b48fed1ca1326443fc359074d12fe04b937e0bdcd6eb5c197"} err="failed to get container status \"3a1fb098837edf1b48fed1ca1326443fc359074d12fe04b937e0bdcd6eb5c197\": rpc error: code = NotFound desc = could not find container \"3a1fb098837edf1b48fed1ca1326443fc359074d12fe04b937e0bdcd6eb5c197\": container with ID starting with 3a1fb098837edf1b48fed1ca1326443fc359074d12fe04b937e0bdcd6eb5c197 not found: ID does not exist"
Oct 30 23:08:35.630506 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.630483    5896 scope.go:122] "RemoveContainer" containerID="de223ede8066e551a8e0f0dc7eddcda58de5c0993b3548ceceee7196345baec9"
Oct 30 23:08:35.630714 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:08:35.630695    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de223ede8066e551a8e0f0dc7eddcda58de5c0993b3548ceceee7196345baec9\": container with ID starting with de223ede8066e551a8e0f0dc7eddcda58de5c0993b3548ceceee7196345baec9 not found: ID does not exist" containerID="de223ede8066e551a8e0f0dc7eddcda58de5c0993b3548ceceee7196345baec9"
Oct 30 23:08:35.630761 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.630722    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de223ede8066e551a8e0f0dc7eddcda58de5c0993b3548ceceee7196345baec9"} err="failed to get container status \"de223ede8066e551a8e0f0dc7eddcda58de5c0993b3548ceceee7196345baec9\": rpc error: code = NotFound desc = could not find container \"de223ede8066e551a8e0f0dc7eddcda58de5c0993b3548ceceee7196345baec9\": container with ID starting with de223ede8066e551a8e0f0dc7eddcda58de5c0993b3548ceceee7196345baec9 not found: ID does not exist"
Oct 30 23:08:35.768030 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.767995    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"]
Oct 30 23:08:35.818914 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:35.818889    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-959ee2f135/isvc-sklearn-mcp-predictor-579ff6755f-xvm66"]
Oct 30 23:08:37.735618 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:37.735584    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c846887e-0c76-4b05-8404-05903f2dd163" path="/var/lib/kubelet/pods/c846887e-0c76-4b05-8404-05903f2dd163/volumes"
Oct 30 23:08:38.362987 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.362954    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"]
Oct 30 23:08:38.363307 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.363294    5896 cpu_manager.go:414] "containerMap: removing container" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kserve-container"
Oct 30 23:08:38.363374 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.363308    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kserve-container"
Oct 30 23:08:38.363374 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.363327    5896 cpu_manager.go:414] "containerMap: removing container" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="storage-initializer"
Oct 30 23:08:38.363374 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.363332    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="storage-initializer"
Oct 30 23:08:38.363374 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.363346    5896 cpu_manager.go:414] "containerMap: removing container" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kube-rbac-proxy"
Oct 30 23:08:38.363374 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.363351    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kube-rbac-proxy"
Oct 30 23:08:38.363374 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.363367    5896 cpu_manager.go:414] "containerMap: removing container" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kserve-agent"
Oct 30 23:08:38.363374 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.363372    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kserve-agent"
Oct 30 23:08:38.363615 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.363424    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kserve-agent"
Oct 30 23:08:38.363615 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.363434    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kserve-container"
Oct 30 23:08:38.363615 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.363443    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="c846887e-0c76-4b05-8404-05903f2dd163" containerName="kube-rbac-proxy"
Oct 30 23:08:38.368162 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.368144    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:38.457426 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.457398    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2w2m\" (UniqueName: \"kubernetes.io/projected/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kube-api-access-h2w2m\") pod \"isvc-pmml-predictor-55f4ff6db9-mp5xp\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") " pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:38.457536 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.457433    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-proxy-tls\") pod \"isvc-pmml-predictor-55f4ff6db9-mp5xp\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") " pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:38.457536 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.457453    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-predictor-55f4ff6db9-mp5xp\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") " pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:38.457613 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.457543    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-cabundle-cert\") pod \"isvc-pmml-predictor-55f4ff6db9-mp5xp\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") " pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:38.457675 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.457608    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kserve-provision-location\") pod \"isvc-pmml-predictor-55f4ff6db9-mp5xp\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") " pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:38.472844 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.472821    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 23:08:38.473042 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.473029    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"kube-root-ca.crt\""
Oct 30 23:08:38.473122 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.473087    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-pmml-predictor-serving-cert\""
Oct 30 23:08:38.473228 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.473106    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"openshift-service-ca.crt\""
Oct 30 23:08:38.473228 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.473151    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-959ee2f135\"/\"isvc-pmml-kube-rbac-proxy-sar-config\""
Oct 30 23:08:38.473228 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.473154    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"default-dockercfg-s9cqw\""
Oct 30 23:08:38.473228 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.473152    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-959ee2f135\"/\"seaweedfs-s3-creds\""
Oct 30 23:08:38.474288 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.474269    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"]
Oct 30 23:08:38.558927 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.558902    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-predictor-55f4ff6db9-mp5xp\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") " pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:38.559034 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.558933    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-cabundle-cert\") pod \"isvc-pmml-predictor-55f4ff6db9-mp5xp\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") " pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:38.559034 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.558978    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kserve-provision-location\") pod \"isvc-pmml-predictor-55f4ff6db9-mp5xp\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") " pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:38.559034 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.559000    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2w2m\" (UniqueName: \"kubernetes.io/projected/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kube-api-access-h2w2m\") pod \"isvc-pmml-predictor-55f4ff6db9-mp5xp\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") " pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:38.559177 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.559104    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-proxy-tls\") pod \"isvc-pmml-predictor-55f4ff6db9-mp5xp\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") " pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:38.559450 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.559427    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kserve-provision-location\") pod \"isvc-pmml-predictor-55f4ff6db9-mp5xp\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") " pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:38.559664 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.559622    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-predictor-55f4ff6db9-mp5xp\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") " pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:38.559735 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.559679    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-cabundle-cert\") pod \"isvc-pmml-predictor-55f4ff6db9-mp5xp\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") " pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:38.561677 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.561655    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-proxy-tls\") pod \"isvc-pmml-predictor-55f4ff6db9-mp5xp\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") " pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:38.622387 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.622337    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2w2m\" (UniqueName: \"kubernetes.io/projected/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kube-api-access-h2w2m\") pod \"isvc-pmml-predictor-55f4ff6db9-mp5xp\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") " pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:38.684725 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:38.684701    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:39.105766 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:39.105744    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"]
Oct 30 23:08:39.108300 ip-10-0-138-202 kubenswrapper[5896]: W1030 23:08:39.108272    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0766bf7c_a630_4d3e_8a2a_6edfe62ccf97.slice/crio-538de742c7ab5122ee2f5800d738eb0bda26504fe29ecb840d81d6da7429cb46 WatchSource:0}: Error finding container 538de742c7ab5122ee2f5800d738eb0bda26504fe29ecb840d81d6da7429cb46: Status 404 returned error can't find the container with id 538de742c7ab5122ee2f5800d738eb0bda26504fe29ecb840d81d6da7429cb46
Oct 30 23:08:39.621245 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:39.621206    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" event={"ID":"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97","Type":"ContainerStarted","Data":"835fffad267ff98ef7c4aed11626a48df24082dd31a046a35482ec63fdc76d5a"}
Oct 30 23:08:39.621383 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:39.621275    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" event={"ID":"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97","Type":"ContainerStarted","Data":"538de742c7ab5122ee2f5800d738eb0bda26504fe29ecb840d81d6da7429cb46"}
Oct 30 23:08:43.640207 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:43.640170    5896 generic.go:356] "Generic (PLEG): container finished" podID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerID="835fffad267ff98ef7c4aed11626a48df24082dd31a046a35482ec63fdc76d5a" exitCode=0
Oct 30 23:08:43.640634 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:43.640243    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" event={"ID":"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97","Type":"ContainerDied","Data":"835fffad267ff98ef7c4aed11626a48df24082dd31a046a35482ec63fdc76d5a"}
Oct 30 23:08:44.226480 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:44.226454    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"]
Oct 30 23:08:44.226717 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:44.226697    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kserve-container" containerID="cri-o://5e74b54d50a61eee050fdc3391a3c47c158acfb13bf876a0137a89ca5b9227aa" gracePeriod=30
Oct 30 23:08:44.226800 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:44.226776    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kube-rbac-proxy" containerID="cri-o://c8cb9c46454e53ab2cc1b3d6d17295f7eea26240890deded58e54437b09b088f" gracePeriod=30
Oct 30 23:08:44.646954 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:44.646866    5896 generic.go:356] "Generic (PLEG): container finished" podID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerID="c8cb9c46454e53ab2cc1b3d6d17295f7eea26240890deded58e54437b09b088f" exitCode=0
Oct 30 23:08:44.646954 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:44.646937    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" event={"ID":"02f326a4-a7ca-4e2e-bf18-f157eade28e6","Type":"ContainerDied","Data":"c8cb9c46454e53ab2cc1b3d6d17295f7eea26240890deded58e54437b09b088f"}
Oct 30 23:08:44.648842 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:44.648821    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" event={"ID":"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97","Type":"ContainerStarted","Data":"2cc5077fe6d03678c4c89f2e64a9054e83cdf2dac141cf4e4b9f335b008bc870"}
Oct 30 23:08:44.648943 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:44.648865    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" event={"ID":"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97","Type":"ContainerStarted","Data":"e3c8b5d1287ea23bf774d50a208ed7d41e44c5d8e6c03ffc1cd3c6bb85b2b8ca"}
Oct 30 23:08:44.649196 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:44.649180    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:44.649348 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:44.649330    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:44.650676 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:44.650634    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.45:8080: connect: connection refused"
Oct 30 23:08:44.772555 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:44.772498    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" podStartSLOduration=2.788192846 podStartE2EDuration="6.772481049s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.984288203s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:08:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:08:44.772329572 +0000 UTC m=+3577.183795538" watchObservedRunningTime="2026-10-30 23:08:44.772481049 +0000 UTC m=+3577.183947016"
Oct 30 23:08:45.217729 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:45.217685    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.44:8643/healthz\": dial tcp 10.134.0.44:8643: connect: connection refused"
Oct 30 23:08:45.653825 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:45.653725    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.45:8080: connect: connection refused"
Oct 30 23:08:47.864624 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:47.864595    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:08:47.933042 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:47.932977    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kube-api-access-gptrd\" (UniqueName: \"kubernetes.io/projected/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kube-api-access-gptrd\") pod \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") "
Oct 30 23:08:47.933042 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:47.933016    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kube-rbac-proxy-sar-config\") pod \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") "
Oct 30 23:08:47.933231 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:47.933054    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/02f326a4-a7ca-4e2e-bf18-f157eade28e6-proxy-tls\" (UniqueName: \"kubernetes.io/secret/02f326a4-a7ca-4e2e-bf18-f157eade28e6-proxy-tls\") pod \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") "
Oct 30 23:08:47.933394 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:47.933368    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kube-rbac-proxy-sar-config" pod "02f326a4-a7ca-4e2e-bf18-f157eade28e6" (UID: "02f326a4-a7ca-4e2e-bf18-f157eade28e6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:08:47.935047 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:47.935025    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kube-api-access-gptrd" pod "02f326a4-a7ca-4e2e-bf18-f157eade28e6" (UID: "02f326a4-a7ca-4e2e-bf18-f157eade28e6"). InnerVolumeSpecName "kube-api-access-gptrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:08:47.935127 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:47.935086    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02f326a4-a7ca-4e2e-bf18-f157eade28e6-proxy-tls" pod "02f326a4-a7ca-4e2e-bf18-f157eade28e6" (UID: "02f326a4-a7ca-4e2e-bf18-f157eade28e6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:08:48.033624 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.033599    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kserve-provision-location\") pod \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") "
Oct 30 23:08:48.033757 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.033662    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/02f326a4-a7ca-4e2e-bf18-f157eade28e6-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/02f326a4-a7ca-4e2e-bf18-f157eade28e6-cabundle-cert\") pod \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\" (UID: \"02f326a4-a7ca-4e2e-bf18-f157eade28e6\") "
Oct 30 23:08:48.033884 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.033869    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gptrd\" (UniqueName: \"kubernetes.io/projected/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kube-api-access-gptrd\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:08:48.033948 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.033890    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:08:48.033948 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.033899    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kserve-provision-location" pod "02f326a4-a7ca-4e2e-bf18-f157eade28e6" (UID: "02f326a4-a7ca-4e2e-bf18-f157eade28e6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:48.033948 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.033905    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/02f326a4-a7ca-4e2e-bf18-f157eade28e6-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:08:48.034058 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.033956    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02f326a4-a7ca-4e2e-bf18-f157eade28e6-cabundle-cert" pod "02f326a4-a7ca-4e2e-bf18-f157eade28e6" (UID: "02f326a4-a7ca-4e2e-bf18-f157eade28e6"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:08:48.134493 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.134464    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/02f326a4-a7ca-4e2e-bf18-f157eade28e6-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:08:48.134493 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.134491    5896 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/02f326a4-a7ca-4e2e-bf18-f157eade28e6-cabundle-cert\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:08:48.669133 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.669058    5896 generic.go:356] "Generic (PLEG): container finished" podID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerID="5e74b54d50a61eee050fdc3391a3c47c158acfb13bf876a0137a89ca5b9227aa" exitCode=0
Oct 30 23:08:48.669267 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.669193    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"
Oct 30 23:08:48.669267 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.669191    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" event={"ID":"02f326a4-a7ca-4e2e-bf18-f157eade28e6","Type":"ContainerDied","Data":"5e74b54d50a61eee050fdc3391a3c47c158acfb13bf876a0137a89ca5b9227aa"}
Oct 30 23:08:48.669342 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.669279    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6" event={"ID":"02f326a4-a7ca-4e2e-bf18-f157eade28e6","Type":"ContainerDied","Data":"3a2702d80d6f03476f1634d85694eaf817a1ca23ca451252c63952e353006f54"}
Oct 30 23:08:48.669342 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.669306    5896 scope.go:122] "RemoveContainer" containerID="c8cb9c46454e53ab2cc1b3d6d17295f7eea26240890deded58e54437b09b088f"
Oct 30 23:08:48.679043 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.679020    5896 scope.go:122] "RemoveContainer" containerID="5e74b54d50a61eee050fdc3391a3c47c158acfb13bf876a0137a89ca5b9227aa"
Oct 30 23:08:48.686359 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.686338    5896 scope.go:122] "RemoveContainer" containerID="79c96433ac99839c32d68489821ffb8cbf21b1b46833dff3ea2690d18fa82fe2"
Oct 30 23:08:48.693553 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.693536    5896 scope.go:122] "RemoveContainer" containerID="c8cb9c46454e53ab2cc1b3d6d17295f7eea26240890deded58e54437b09b088f"
Oct 30 23:08:48.693837 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:08:48.693818    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8cb9c46454e53ab2cc1b3d6d17295f7eea26240890deded58e54437b09b088f\": container with ID starting with c8cb9c46454e53ab2cc1b3d6d17295f7eea26240890deded58e54437b09b088f not found: ID does not exist" containerID="c8cb9c46454e53ab2cc1b3d6d17295f7eea26240890deded58e54437b09b088f"
Oct 30 23:08:48.693899 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.693844    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8cb9c46454e53ab2cc1b3d6d17295f7eea26240890deded58e54437b09b088f"} err="failed to get container status \"c8cb9c46454e53ab2cc1b3d6d17295f7eea26240890deded58e54437b09b088f\": rpc error: code = NotFound desc = could not find container \"c8cb9c46454e53ab2cc1b3d6d17295f7eea26240890deded58e54437b09b088f\": container with ID starting with c8cb9c46454e53ab2cc1b3d6d17295f7eea26240890deded58e54437b09b088f not found: ID does not exist"
Oct 30 23:08:48.693899 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.693861    5896 scope.go:122] "RemoveContainer" containerID="5e74b54d50a61eee050fdc3391a3c47c158acfb13bf876a0137a89ca5b9227aa"
Oct 30 23:08:48.694107 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:08:48.694091    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e74b54d50a61eee050fdc3391a3c47c158acfb13bf876a0137a89ca5b9227aa\": container with ID starting with 5e74b54d50a61eee050fdc3391a3c47c158acfb13bf876a0137a89ca5b9227aa not found: ID does not exist" containerID="5e74b54d50a61eee050fdc3391a3c47c158acfb13bf876a0137a89ca5b9227aa"
Oct 30 23:08:48.694151 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.694123    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e74b54d50a61eee050fdc3391a3c47c158acfb13bf876a0137a89ca5b9227aa"} err="failed to get container status \"5e74b54d50a61eee050fdc3391a3c47c158acfb13bf876a0137a89ca5b9227aa\": rpc error: code = NotFound desc = could not find container \"5e74b54d50a61eee050fdc3391a3c47c158acfb13bf876a0137a89ca5b9227aa\": container with ID starting with 5e74b54d50a61eee050fdc3391a3c47c158acfb13bf876a0137a89ca5b9227aa not found: ID does not exist"
Oct 30 23:08:48.694151 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.694139    5896 scope.go:122] "RemoveContainer" containerID="79c96433ac99839c32d68489821ffb8cbf21b1b46833dff3ea2690d18fa82fe2"
Oct 30 23:08:48.694359 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:08:48.694345    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79c96433ac99839c32d68489821ffb8cbf21b1b46833dff3ea2690d18fa82fe2\": container with ID starting with 79c96433ac99839c32d68489821ffb8cbf21b1b46833dff3ea2690d18fa82fe2 not found: ID does not exist" containerID="79c96433ac99839c32d68489821ffb8cbf21b1b46833dff3ea2690d18fa82fe2"
Oct 30 23:08:48.694407 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.694362    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79c96433ac99839c32d68489821ffb8cbf21b1b46833dff3ea2690d18fa82fe2"} err="failed to get container status \"79c96433ac99839c32d68489821ffb8cbf21b1b46833dff3ea2690d18fa82fe2\": rpc error: code = NotFound desc = could not find container \"79c96433ac99839c32d68489821ffb8cbf21b1b46833dff3ea2690d18fa82fe2\": container with ID starting with 79c96433ac99839c32d68489821ffb8cbf21b1b46833dff3ea2690d18fa82fe2 not found: ID does not exist"
Oct 30 23:08:48.889183 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.889157    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"]
Oct 30 23:08:48.894537 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:48.894516    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-bc5112d610/isvc-pmml-runtime-predictor-5f474bd489-2wfv6"]
Oct 30 23:08:49.734493 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:49.734459    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" path="/var/lib/kubelet/pods/02f326a4-a7ca-4e2e-bf18-f157eade28e6/volumes"
Oct 30 23:08:50.658025 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:50.658000    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:08:50.658460 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:08:50.658437    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.45:8080: connect: connection refused"
Oct 30 23:09:00.659059 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:09:00.659022    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.45:8080: connect: connection refused"
Oct 30 23:09:02.627285 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:09:02.627242    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:09:08.108166 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:09:08.108054    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 23:09:08.124869 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:09:08.115035    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcjjm_10c86ffc-dbfb-48d8-a225-edc41c719c6f/ovn-acl-logging/0.log"
Oct 30 23:09:10.659264 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:09:10.659231    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.45:8080: connect: connection refused"
Oct 30 23:09:20.658904 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:09:20.658819    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.45:8080: connect: connection refused"
Oct 30 23:09:30.659156 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:09:30.659121    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.45:8080: connect: connection refused"
Oct 30 23:09:32.628569 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:09:32.628528    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:09:40.658819 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:09:40.658779    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.45:8080: connect: connection refused"
Oct 30 23:09:50.658847 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:09:50.658812    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.45:8080: connect: connection refused"
Oct 30 23:10:00.658432 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:00.658395    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.45:8080: connect: connection refused"
Oct 30 23:10:02.625480 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:10:02.625446    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:10:06.731772 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:06.731748    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:10:20.149516 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:20.149486    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"]
Oct 30 23:10:20.149955 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:20.149743    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kserve-container" containerID="cri-o://e3c8b5d1287ea23bf774d50a208ed7d41e44c5d8e6c03ffc1cd3c6bb85b2b8ca" gracePeriod=30
Oct 30 23:10:20.149955 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:20.149780    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kube-rbac-proxy" containerID="cri-o://2cc5077fe6d03678c4c89f2e64a9054e83cdf2dac141cf4e4b9f335b008bc870" gracePeriod=30
Oct 30 23:10:20.654964 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:20.654925    5896 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.45:8643/healthz\": dial tcp 10.134.0.45:8643: connect: connection refused"
Oct 30 23:10:21.034509 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:21.034477    5896 generic.go:356] "Generic (PLEG): container finished" podID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerID="2cc5077fe6d03678c4c89f2e64a9054e83cdf2dac141cf4e4b9f335b008bc870" exitCode=0
Oct 30 23:10:21.034690 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:21.034527    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" event={"ID":"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97","Type":"ContainerDied","Data":"2cc5077fe6d03678c4c89f2e64a9054e83cdf2dac141cf4e4b9f335b008bc870"}
Oct 30 23:10:23.892337 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:23.892313    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:10:23.977709 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:23.977687    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-proxy-tls\" (UniqueName: \"kubernetes.io/secret/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-proxy-tls\") pod \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") "
Oct 30 23:10:23.977852 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:23.977741    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kube-rbac-proxy-sar-config\") pod \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") "
Oct 30 23:10:23.977852 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:23.977816    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-cabundle-cert\") pod \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") "
Oct 30 23:10:23.977964 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:23.977879    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kube-api-access-h2w2m\" (UniqueName: \"kubernetes.io/projected/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kube-api-access-h2w2m\") pod \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") "
Oct 30 23:10:23.977964 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:23.977928    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kserve-provision-location\") pod \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\" (UID: \"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97\") "
Oct 30 23:10:23.978127 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:23.978102    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kube-rbac-proxy-sar-config" pod "0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" (UID: "0766bf7c-a630-4d3e-8a2a-6edfe62ccf97"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:10:23.978314 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:23.978285    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-cabundle-cert" pod "0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" (UID: "0766bf7c-a630-4d3e-8a2a-6edfe62ccf97"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:10:23.978314 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:23.978288    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kserve-provision-location" pod "0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" (UID: "0766bf7c-a630-4d3e-8a2a-6edfe62ccf97"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:10:23.979898 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:23.979878    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-proxy-tls" pod "0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" (UID: "0766bf7c-a630-4d3e-8a2a-6edfe62ccf97"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:10:23.980093 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:23.980069    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kube-api-access-h2w2m" pod "0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" (UID: "0766bf7c-a630-4d3e-8a2a-6edfe62ccf97"). InnerVolumeSpecName "kube-api-access-h2w2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:10:24.049500 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.049471    5896 generic.go:356] "Generic (PLEG): container finished" podID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerID="e3c8b5d1287ea23bf774d50a208ed7d41e44c5d8e6c03ffc1cd3c6bb85b2b8ca" exitCode=0
Oct 30 23:10:24.049606 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.049585    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"
Oct 30 23:10:24.049693 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.049582    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" event={"ID":"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97","Type":"ContainerDied","Data":"e3c8b5d1287ea23bf774d50a208ed7d41e44c5d8e6c03ffc1cd3c6bb85b2b8ca"}
Oct 30 23:10:24.049755 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.049731    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp" event={"ID":"0766bf7c-a630-4d3e-8a2a-6edfe62ccf97","Type":"ContainerDied","Data":"538de742c7ab5122ee2f5800d738eb0bda26504fe29ecb840d81d6da7429cb46"}
Oct 30 23:10:24.049806 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.049774    5896 scope.go:122] "RemoveContainer" containerID="2cc5077fe6d03678c4c89f2e64a9054e83cdf2dac141cf4e4b9f335b008bc870"
Oct 30 23:10:24.058106 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.058086    5896 scope.go:122] "RemoveContainer" containerID="e3c8b5d1287ea23bf774d50a208ed7d41e44c5d8e6c03ffc1cd3c6bb85b2b8ca"
Oct 30 23:10:24.066135 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.065906    5896 scope.go:122] "RemoveContainer" containerID="835fffad267ff98ef7c4aed11626a48df24082dd31a046a35482ec63fdc76d5a"
Oct 30 23:10:24.075534 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.075517    5896 scope.go:122] "RemoveContainer" containerID="2cc5077fe6d03678c4c89f2e64a9054e83cdf2dac141cf4e4b9f335b008bc870"
Oct 30 23:10:24.075839 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:10:24.075813    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cc5077fe6d03678c4c89f2e64a9054e83cdf2dac141cf4e4b9f335b008bc870\": container with ID starting with 2cc5077fe6d03678c4c89f2e64a9054e83cdf2dac141cf4e4b9f335b008bc870 not found: ID does not exist" containerID="2cc5077fe6d03678c4c89f2e64a9054e83cdf2dac141cf4e4b9f335b008bc870"
Oct 30 23:10:24.075908 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.075852    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cc5077fe6d03678c4c89f2e64a9054e83cdf2dac141cf4e4b9f335b008bc870"} err="failed to get container status \"2cc5077fe6d03678c4c89f2e64a9054e83cdf2dac141cf4e4b9f335b008bc870\": rpc error: code = NotFound desc = could not find container \"2cc5077fe6d03678c4c89f2e64a9054e83cdf2dac141cf4e4b9f335b008bc870\": container with ID starting with 2cc5077fe6d03678c4c89f2e64a9054e83cdf2dac141cf4e4b9f335b008bc870 not found: ID does not exist"
Oct 30 23:10:24.075908 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.075875    5896 scope.go:122] "RemoveContainer" containerID="e3c8b5d1287ea23bf774d50a208ed7d41e44c5d8e6c03ffc1cd3c6bb85b2b8ca"
Oct 30 23:10:24.076172 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:10:24.076148    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3c8b5d1287ea23bf774d50a208ed7d41e44c5d8e6c03ffc1cd3c6bb85b2b8ca\": container with ID starting with e3c8b5d1287ea23bf774d50a208ed7d41e44c5d8e6c03ffc1cd3c6bb85b2b8ca not found: ID does not exist" containerID="e3c8b5d1287ea23bf774d50a208ed7d41e44c5d8e6c03ffc1cd3c6bb85b2b8ca"
Oct 30 23:10:24.076228 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.076182    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3c8b5d1287ea23bf774d50a208ed7d41e44c5d8e6c03ffc1cd3c6bb85b2b8ca"} err="failed to get container status \"e3c8b5d1287ea23bf774d50a208ed7d41e44c5d8e6c03ffc1cd3c6bb85b2b8ca\": rpc error: code = NotFound desc = could not find container \"e3c8b5d1287ea23bf774d50a208ed7d41e44c5d8e6c03ffc1cd3c6bb85b2b8ca\": container with ID starting with e3c8b5d1287ea23bf774d50a208ed7d41e44c5d8e6c03ffc1cd3c6bb85b2b8ca not found: ID does not exist"
Oct 30 23:10:24.076228 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.076205    5896 scope.go:122] "RemoveContainer" containerID="835fffad267ff98ef7c4aed11626a48df24082dd31a046a35482ec63fdc76d5a"
Oct 30 23:10:24.076484 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:10:24.076467    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"835fffad267ff98ef7c4aed11626a48df24082dd31a046a35482ec63fdc76d5a\": container with ID starting with 835fffad267ff98ef7c4aed11626a48df24082dd31a046a35482ec63fdc76d5a not found: ID does not exist" containerID="835fffad267ff98ef7c4aed11626a48df24082dd31a046a35482ec63fdc76d5a"
Oct 30 23:10:24.076547 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.076489    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"835fffad267ff98ef7c4aed11626a48df24082dd31a046a35482ec63fdc76d5a"} err="failed to get container status \"835fffad267ff98ef7c4aed11626a48df24082dd31a046a35482ec63fdc76d5a\": rpc error: code = NotFound desc = could not find container \"835fffad267ff98ef7c4aed11626a48df24082dd31a046a35482ec63fdc76d5a\": container with ID starting with 835fffad267ff98ef7c4aed11626a48df24082dd31a046a35482ec63fdc76d5a not found: ID does not exist"
Oct 30 23:10:24.078861 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.078844    5896 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-cabundle-cert\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:10:24.078921 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.078864    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2w2m\" (UniqueName: \"kubernetes.io/projected/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kube-api-access-h2w2m\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:10:24.078921 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.078875    5896 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kserve-provision-location\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:10:24.078921 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.078884    5896 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-proxy-tls\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:10:24.078921 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.078892    5896 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:10:24.215402 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.215375    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"]
Oct 30 23:10:24.220970 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:24.220946    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-959ee2f135/isvc-pmml-predictor-55f4ff6db9-mp5xp"]
Oct 30 23:10:25.735277 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:25.735240    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" path="/var/lib/kubelet/pods/0766bf7c-a630-4d3e-8a2a-6edfe62ccf97/volumes"
Oct 30 23:10:26.616289 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616255    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kxtcl/must-gather-zvgvf"]
Oct 30 23:10:26.616616 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616603    5896 cpu_manager.go:414] "containerMap: removing container" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kserve-container"
Oct 30 23:10:26.616693 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616617    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kserve-container"
Oct 30 23:10:26.616693 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616634    5896 cpu_manager.go:414] "containerMap: removing container" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kserve-container"
Oct 30 23:10:26.616693 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616654    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kserve-container"
Oct 30 23:10:26.616693 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616675    5896 cpu_manager.go:414] "containerMap: removing container" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kube-rbac-proxy"
Oct 30 23:10:26.616693 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616680    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kube-rbac-proxy"
Oct 30 23:10:26.616693 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616694    5896 cpu_manager.go:414] "containerMap: removing container" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kube-rbac-proxy"
Oct 30 23:10:26.616880 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616699    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kube-rbac-proxy"
Oct 30 23:10:26.616880 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616706    5896 cpu_manager.go:414] "containerMap: removing container" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="storage-initializer"
Oct 30 23:10:26.616880 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616711    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="storage-initializer"
Oct 30 23:10:26.616880 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616717    5896 cpu_manager.go:414] "containerMap: removing container" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="storage-initializer"
Oct 30 23:10:26.616880 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616721    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="storage-initializer"
Oct 30 23:10:26.616880 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616766    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kserve-container"
Oct 30 23:10:26.616880 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616777    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="02f326a4-a7ca-4e2e-bf18-f157eade28e6" containerName="kube-rbac-proxy"
Oct 30 23:10:26.616880 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616783    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kube-rbac-proxy"
Oct 30 23:10:26.616880 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.616789    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="0766bf7c-a630-4d3e-8a2a-6edfe62ccf97" containerName="kserve-container"
Oct 30 23:10:26.619843 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.619827    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kxtcl/must-gather-zvgvf"
Oct 30 23:10:26.698989 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.698960    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a5f4f548-eeba-4694-a7a4-7f4ad1f1177b-must-gather-output\") pod \"must-gather-zvgvf\" (UID: \"a5f4f548-eeba-4694-a7a4-7f4ad1f1177b\") " pod="openshift-must-gather-kxtcl/must-gather-zvgvf"
Oct 30 23:10:26.699112 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.698993    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6qgh\" (UniqueName: \"kubernetes.io/projected/a5f4f548-eeba-4694-a7a4-7f4ad1f1177b-kube-api-access-g6qgh\") pod \"must-gather-zvgvf\" (UID: \"a5f4f548-eeba-4694-a7a4-7f4ad1f1177b\") " pod="openshift-must-gather-kxtcl/must-gather-zvgvf"
Oct 30 23:10:26.724338 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.724317    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kxtcl\"/\"openshift-service-ca.crt\""
Oct 30 23:10:26.724338 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.724329    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kxtcl\"/\"default-dockercfg-9njd4\""
Oct 30 23:10:26.724487 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.724355    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kxtcl\"/\"kube-root-ca.crt\""
Oct 30 23:10:26.725631 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.725611    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kxtcl/must-gather-zvgvf"]
Oct 30 23:10:26.799575 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.799548    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a5f4f548-eeba-4694-a7a4-7f4ad1f1177b-must-gather-output\") pod \"must-gather-zvgvf\" (UID: \"a5f4f548-eeba-4694-a7a4-7f4ad1f1177b\") " pod="openshift-must-gather-kxtcl/must-gather-zvgvf"
Oct 30 23:10:26.799935 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.799588    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6qgh\" (UniqueName: \"kubernetes.io/projected/a5f4f548-eeba-4694-a7a4-7f4ad1f1177b-kube-api-access-g6qgh\") pod \"must-gather-zvgvf\" (UID: \"a5f4f548-eeba-4694-a7a4-7f4ad1f1177b\") " pod="openshift-must-gather-kxtcl/must-gather-zvgvf"
Oct 30 23:10:26.800012 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.799993    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a5f4f548-eeba-4694-a7a4-7f4ad1f1177b-must-gather-output\") pod \"must-gather-zvgvf\" (UID: \"a5f4f548-eeba-4694-a7a4-7f4ad1f1177b\") " pod="openshift-must-gather-kxtcl/must-gather-zvgvf"
Oct 30 23:10:26.862960 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.862939    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6qgh\" (UniqueName: \"kubernetes.io/projected/a5f4f548-eeba-4694-a7a4-7f4ad1f1177b-kube-api-access-g6qgh\") pod \"must-gather-zvgvf\" (UID: \"a5f4f548-eeba-4694-a7a4-7f4ad1f1177b\") " pod="openshift-must-gather-kxtcl/must-gather-zvgvf"
Oct 30 23:10:26.948698 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:26.948648    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kxtcl/must-gather-zvgvf"
Oct 30 23:10:27.321184 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:27.321123    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kxtcl/must-gather-zvgvf"]
Oct 30 23:10:27.323803 ip-10-0-138-202 kubenswrapper[5896]: W1030 23:10:27.323774    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5f4f548_eeba_4694_a7a4_7f4ad1f1177b.slice/crio-60d2df50ce12e62f9d55254d074d41a91445287cff191985b9c3fdb9a44fe608 WatchSource:0}: Error finding container 60d2df50ce12e62f9d55254d074d41a91445287cff191985b9c3fdb9a44fe608: Status 404 returned error can't find the container with id 60d2df50ce12e62f9d55254d074d41a91445287cff191985b9c3fdb9a44fe608
Oct 30 23:10:27.325690 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:27.325674    5896 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 23:10:28.067499 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:28.067464    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kxtcl/must-gather-zvgvf" event={"ID":"a5f4f548-eeba-4694-a7a4-7f4ad1f1177b","Type":"ContainerStarted","Data":"60d2df50ce12e62f9d55254d074d41a91445287cff191985b9c3fdb9a44fe608"}
Oct 30 23:10:32.632166 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:10:32.632110    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:10:35.100071 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:35.100036    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kxtcl/must-gather-zvgvf" event={"ID":"a5f4f548-eeba-4694-a7a4-7f4ad1f1177b","Type":"ContainerStarted","Data":"54287cb9ff6103f1092696a7ce7438bb9f234fb82a0e4f77f10893f212f42fb1"}
Oct 30 23:10:35.100071 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:35.100077    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kxtcl/must-gather-zvgvf" event={"ID":"a5f4f548-eeba-4694-a7a4-7f4ad1f1177b","Type":"ContainerStarted","Data":"6ae41026648c62348cef53c19d706524f63f27504289f1a0000164892f21e724"}
Oct 30 23:10:35.218060 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:10:35.218008    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-kxtcl/must-gather-zvgvf" podStartSLOduration=2.339128659 podStartE2EDuration="9.21799459s" totalImagesPullingTime="6.878865931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:10:35.217085211 +0000 UTC m=+3687.628551177" watchObservedRunningTime="2026-10-30 23:10:35.21799459 +0000 UTC m=+3687.629460557"
Oct 30 23:11:02.625875 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:11:02.625790    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:11:32.629790 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:11:32.629756    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:11:40.378414 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:40.378383    5896 generic.go:356] "Generic (PLEG): container finished" podID="a5f4f548-eeba-4694-a7a4-7f4ad1f1177b" containerID="6ae41026648c62348cef53c19d706524f63f27504289f1a0000164892f21e724" exitCode=0
Oct 30 23:11:40.378837 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:40.378458    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kxtcl/must-gather-zvgvf" event={"ID":"a5f4f548-eeba-4694-a7a4-7f4ad1f1177b","Type":"ContainerDied","Data":"6ae41026648c62348cef53c19d706524f63f27504289f1a0000164892f21e724"}
Oct 30 23:11:40.378837 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:40.378769    5896 scope.go:122] "RemoveContainer" containerID="6ae41026648c62348cef53c19d706524f63f27504289f1a0000164892f21e724"
Oct 30 23:11:41.349613 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:41.349585    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-kxtcl_must-gather-zvgvf_a5f4f548-eeba-4694-a7a4-7f4ad1f1177b/gather/0.log"
Oct 30 23:11:47.736144 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:47.736112    5896 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-kxtcl/must-gather-zvgvf"]
Oct 30 23:11:47.736546 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:47.736289    5896 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-must-gather-kxtcl/must-gather-zvgvf" podUID="a5f4f548-eeba-4694-a7a4-7f4ad1f1177b" containerName="copy" containerID="cri-o://54287cb9ff6103f1092696a7ce7438bb9f234fb82a0e4f77f10893f212f42fb1" gracePeriod=2
Oct 30 23:11:47.786629 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:47.786597    5896 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-kxtcl/must-gather-zvgvf"]
Oct 30 23:11:47.788417 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:11:47.788385    5896 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-zvgvf\" is forbidden: User \"system:node:ip-10-0-138-202.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-kxtcl\": no relationship found between node 'ip-10-0-138-202.us-west-1.compute.internal' and this object" podUID="a5f4f548-eeba-4694-a7a4-7f4ad1f1177b" pod="openshift-must-gather-kxtcl/must-gather-zvgvf"
Oct 30 23:11:48.014929 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.014909    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-kxtcl_must-gather-zvgvf_a5f4f548-eeba-4694-a7a4-7f4ad1f1177b/copy/0.log"
Oct 30 23:11:48.017379 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.017361    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kxtcl/must-gather-zvgvf"
Oct 30 23:11:48.069545 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:11:48.069516    5896 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-zvgvf\" is forbidden: User \"system:node:ip-10-0-138-202.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-kxtcl\": no relationship found between node 'ip-10-0-138-202.us-west-1.compute.internal' and this object" podUID="a5f4f548-eeba-4694-a7a4-7f4ad1f1177b" pod="openshift-must-gather-kxtcl/must-gather-zvgvf"
Oct 30 23:11:48.102880 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.102858    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5f4f548-eeba-4694-a7a4-7f4ad1f1177b-must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a5f4f548-eeba-4694-a7a4-7f4ad1f1177b-must-gather-output\") pod \"a5f4f548-eeba-4694-a7a4-7f4ad1f1177b\" (UID: \"a5f4f548-eeba-4694-a7a4-7f4ad1f1177b\") "
Oct 30 23:11:48.102952 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.102939    5896 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5f4f548-eeba-4694-a7a4-7f4ad1f1177b-kube-api-access-g6qgh\" (UniqueName: \"kubernetes.io/projected/a5f4f548-eeba-4694-a7a4-7f4ad1f1177b-kube-api-access-g6qgh\") pod \"a5f4f548-eeba-4694-a7a4-7f4ad1f1177b\" (UID: \"a5f4f548-eeba-4694-a7a4-7f4ad1f1177b\") "
Oct 30 23:11:48.104089 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.104066    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f4f548-eeba-4694-a7a4-7f4ad1f1177b-must-gather-output" pod "a5f4f548-eeba-4694-a7a4-7f4ad1f1177b" (UID: "a5f4f548-eeba-4694-a7a4-7f4ad1f1177b"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:11:48.105095 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.105077    5896 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5f4f548-eeba-4694-a7a4-7f4ad1f1177b-kube-api-access-g6qgh" pod "a5f4f548-eeba-4694-a7a4-7f4ad1f1177b" (UID: "a5f4f548-eeba-4694-a7a4-7f4ad1f1177b"). InnerVolumeSpecName "kube-api-access-g6qgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:11:48.203905 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.203887    5896 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a5f4f548-eeba-4694-a7a4-7f4ad1f1177b-must-gather-output\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:11:48.203905 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.203906    5896 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6qgh\" (UniqueName: \"kubernetes.io/projected/a5f4f548-eeba-4694-a7a4-7f4ad1f1177b-kube-api-access-g6qgh\") on node \"ip-10-0-138-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 23:11:48.411785 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.411736    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-kxtcl_must-gather-zvgvf_a5f4f548-eeba-4694-a7a4-7f4ad1f1177b/copy/0.log"
Oct 30 23:11:48.412119 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.412101    5896 generic.go:356] "Generic (PLEG): container finished" podID="a5f4f548-eeba-4694-a7a4-7f4ad1f1177b" containerID="54287cb9ff6103f1092696a7ce7438bb9f234fb82a0e4f77f10893f212f42fb1" exitCode=143
Oct 30 23:11:48.412196 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.412166    5896 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kxtcl/must-gather-zvgvf"
Oct 30 23:11:48.412257 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.412170    5896 scope.go:122] "RemoveContainer" containerID="54287cb9ff6103f1092696a7ce7438bb9f234fb82a0e4f77f10893f212f42fb1"
Oct 30 23:11:48.420479 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.420459    5896 scope.go:122] "RemoveContainer" containerID="6ae41026648c62348cef53c19d706524f63f27504289f1a0000164892f21e724"
Oct 30 23:11:48.432438 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.432418    5896 scope.go:122] "RemoveContainer" containerID="54287cb9ff6103f1092696a7ce7438bb9f234fb82a0e4f77f10893f212f42fb1"
Oct 30 23:11:48.432720 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:11:48.432703    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54287cb9ff6103f1092696a7ce7438bb9f234fb82a0e4f77f10893f212f42fb1\": container with ID starting with 54287cb9ff6103f1092696a7ce7438bb9f234fb82a0e4f77f10893f212f42fb1 not found: ID does not exist" containerID="54287cb9ff6103f1092696a7ce7438bb9f234fb82a0e4f77f10893f212f42fb1"
Oct 30 23:11:48.432788 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.432729    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54287cb9ff6103f1092696a7ce7438bb9f234fb82a0e4f77f10893f212f42fb1"} err="failed to get container status \"54287cb9ff6103f1092696a7ce7438bb9f234fb82a0e4f77f10893f212f42fb1\": rpc error: code = NotFound desc = could not find container \"54287cb9ff6103f1092696a7ce7438bb9f234fb82a0e4f77f10893f212f42fb1\": container with ID starting with 54287cb9ff6103f1092696a7ce7438bb9f234fb82a0e4f77f10893f212f42fb1 not found: ID does not exist"
Oct 30 23:11:48.432788 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.432745    5896 scope.go:122] "RemoveContainer" containerID="6ae41026648c62348cef53c19d706524f63f27504289f1a0000164892f21e724"
Oct 30 23:11:48.432952 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:11:48.432936    5896 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ae41026648c62348cef53c19d706524f63f27504289f1a0000164892f21e724\": container with ID starting with 6ae41026648c62348cef53c19d706524f63f27504289f1a0000164892f21e724 not found: ID does not exist" containerID="6ae41026648c62348cef53c19d706524f63f27504289f1a0000164892f21e724"
Oct 30 23:11:48.432996 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:48.432959    5896 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ae41026648c62348cef53c19d706524f63f27504289f1a0000164892f21e724"} err="failed to get container status \"6ae41026648c62348cef53c19d706524f63f27504289f1a0000164892f21e724\": rpc error: code = NotFound desc = could not find container \"6ae41026648c62348cef53c19d706524f63f27504289f1a0000164892f21e724\": container with ID starting with 6ae41026648c62348cef53c19d706524f63f27504289f1a0000164892f21e724 not found: ID does not exist"
Oct 30 23:11:48.464221 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:11:48.464191    5896 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-zvgvf\" is forbidden: User \"system:node:ip-10-0-138-202.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-kxtcl\": no relationship found between node 'ip-10-0-138-202.us-west-1.compute.internal' and this object" podUID="a5f4f548-eeba-4694-a7a4-7f4ad1f1177b" pod="openshift-must-gather-kxtcl/must-gather-zvgvf"
Oct 30 23:11:48.515854 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:11:48.515818    5896 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-zvgvf\" is forbidden: User \"system:node:ip-10-0-138-202.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-kxtcl\": no relationship found between node 'ip-10-0-138-202.us-west-1.compute.internal' and this object" podUID="a5f4f548-eeba-4694-a7a4-7f4ad1f1177b" pod="openshift-must-gather-kxtcl/must-gather-zvgvf"
Oct 30 23:11:49.735288 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:49.735256    5896 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5f4f548-eeba-4694-a7a4-7f4ad1f1177b" path="/var/lib/kubelet/pods/a5f4f548-eeba-4694-a7a4-7f4ad1f1177b/volumes"
Oct 30 23:11:52.084969 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:52.084942    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jxjd5_796116fb-6bb5-4c50-8b4f-9aabe94ea8ce/global-pull-secret-syncer/0.log"
Oct 30 23:11:56.133598 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:56.133563    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-f4lgs_653cd0d6-b7a2-446a-b677-406e3e8e56ce/konnectivity-agent/0.log"
Oct 30 23:11:57.867205 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:11:57.867173    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-202.us-west-1.compute.internal_dc53250de6efc08206be074330c1cea0/haproxy/0.log"
Oct 30 23:12:01.946963 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:01.946938    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a15ec80d-1d69-4956-b781-03a41fa68f7d/alertmanager/0.log"
Oct 30 23:12:02.377166 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:02.377142    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a15ec80d-1d69-4956-b781-03a41fa68f7d/config-reloader/0.log"
Oct 30 23:12:02.626247 ip-10-0-138-202 kubenswrapper[5896]: E1030 23:12:02.626210    5896 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-dmwlw" containerName="extractor"
Oct 30 23:12:02.801153 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:02.801127    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a15ec80d-1d69-4956-b781-03a41fa68f7d/kube-rbac-proxy-web/0.log"
Oct 30 23:12:03.218517 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:03.218495    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a15ec80d-1d69-4956-b781-03a41fa68f7d/kube-rbac-proxy/0.log"
Oct 30 23:12:03.637629 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:03.637554    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a15ec80d-1d69-4956-b781-03a41fa68f7d/kube-rbac-proxy-metric/0.log"
Oct 30 23:12:04.056661 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:04.056614    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a15ec80d-1d69-4956-b781-03a41fa68f7d/prom-label-proxy/0.log"
Oct 30 23:12:04.496611 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:04.496591    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a15ec80d-1d69-4956-b781-03a41fa68f7d/init-config-reloader/0.log"
Oct 30 23:12:05.352963 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:05.352909    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-sz4mk_6d74e018-659b-4322-a280-b207fed7271f/kube-state-metrics/0.log"
Oct 30 23:12:05.768196 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:05.768172    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-sz4mk_6d74e018-659b-4322-a280-b207fed7271f/kube-rbac-proxy-main/0.log"
Oct 30 23:12:05.968233 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:05.968211    5896 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"]
Oct 30 23:12:05.968521 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:05.968510    5896 cpu_manager.go:414] "containerMap: removing container" podUID="a5f4f548-eeba-4694-a7a4-7f4ad1f1177b" containerName="gather"
Oct 30 23:12:05.968562 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:05.968522    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5f4f548-eeba-4694-a7a4-7f4ad1f1177b" containerName="gather"
Oct 30 23:12:05.968562 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:05.968541    5896 cpu_manager.go:414] "containerMap: removing container" podUID="a5f4f548-eeba-4694-a7a4-7f4ad1f1177b" containerName="copy"
Oct 30 23:12:05.968562 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:05.968546    5896 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5f4f548-eeba-4694-a7a4-7f4ad1f1177b" containerName="copy"
Oct 30 23:12:05.968688 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:05.968608    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5f4f548-eeba-4694-a7a4-7f4ad1f1177b" containerName="gather"
Oct 30 23:12:05.968688 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:05.968617    5896 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5f4f548-eeba-4694-a7a4-7f4ad1f1177b" containerName="copy"
Oct 30 23:12:05.973392 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:05.973375    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.027385 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.027335    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dk65g\"/\"openshift-service-ca.crt\""
Oct 30 23:12:06.027385 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.027364    5896 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dk65g\"/\"default-dockercfg-w2m2f\""
Oct 30 23:12:06.038541 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.038517    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f7dc650-8b4a-4af1-ae9c-1936ab8b4635-podres\") pod \"perf-node-gather-daemonset-87h5g\" (UID: \"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635\") " pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.038630 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.038554    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f7dc650-8b4a-4af1-ae9c-1936ab8b4635-sys\") pod \"perf-node-gather-daemonset-87h5g\" (UID: \"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635\") " pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.038630 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.038582    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg7gn\" (UniqueName: \"kubernetes.io/projected/6f7dc650-8b4a-4af1-ae9c-1936ab8b4635-kube-api-access-kg7gn\") pod \"perf-node-gather-daemonset-87h5g\" (UID: \"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635\") " pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.038745 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.038652    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f7dc650-8b4a-4af1-ae9c-1936ab8b4635-lib-modules\") pod \"perf-node-gather-daemonset-87h5g\" (UID: \"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635\") " pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.038780 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.038757    5896 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f7dc650-8b4a-4af1-ae9c-1936ab8b4635-proc\") pod \"perf-node-gather-daemonset-87h5g\" (UID: \"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635\") " pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.077800 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.077781    5896 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dk65g\"/\"kube-root-ca.crt\""
Oct 30 23:12:06.078801 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.078782    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"]
Oct 30 23:12:06.139984 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.139962    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f7dc650-8b4a-4af1-ae9c-1936ab8b4635-proc\") pod \"perf-node-gather-daemonset-87h5g\" (UID: \"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635\") " pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.140116 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.140004    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f7dc650-8b4a-4af1-ae9c-1936ab8b4635-podres\") pod \"perf-node-gather-daemonset-87h5g\" (UID: \"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635\") " pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.140116 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.140023    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f7dc650-8b4a-4af1-ae9c-1936ab8b4635-sys\") pod \"perf-node-gather-daemonset-87h5g\" (UID: \"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635\") " pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.140116 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.140043    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg7gn\" (UniqueName: \"kubernetes.io/projected/6f7dc650-8b4a-4af1-ae9c-1936ab8b4635-kube-api-access-kg7gn\") pod \"perf-node-gather-daemonset-87h5g\" (UID: \"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635\") " pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.140116 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.140081    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f7dc650-8b4a-4af1-ae9c-1936ab8b4635-proc\") pod \"perf-node-gather-daemonset-87h5g\" (UID: \"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635\") " pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.140116 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.140086    5896 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f7dc650-8b4a-4af1-ae9c-1936ab8b4635-lib-modules\") pod \"perf-node-gather-daemonset-87h5g\" (UID: \"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635\") " pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.140116 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.140101    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f7dc650-8b4a-4af1-ae9c-1936ab8b4635-sys\") pod \"perf-node-gather-daemonset-87h5g\" (UID: \"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635\") " pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.140116 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.140103    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f7dc650-8b4a-4af1-ae9c-1936ab8b4635-podres\") pod \"perf-node-gather-daemonset-87h5g\" (UID: \"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635\") " pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.140378 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.140162    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f7dc650-8b4a-4af1-ae9c-1936ab8b4635-lib-modules\") pod \"perf-node-gather-daemonset-87h5g\" (UID: \"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635\") " pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.186417 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.186395    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-sz4mk_6d74e018-659b-4322-a280-b207fed7271f/kube-rbac-proxy-self/0.log"
Oct 30 23:12:06.203707 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.203688    5896 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg7gn\" (UniqueName: \"kubernetes.io/projected/6f7dc650-8b4a-4af1-ae9c-1936ab8b4635-kube-api-access-kg7gn\") pod \"perf-node-gather-daemonset-87h5g\" (UID: \"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635\") " pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.285354 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.285310    5896 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:06.624545 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.624471    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7474bd8d9b-c28kq_09cc5ce9-e655-4872-a076-43e6686040e8/metrics-server/0.log"
Oct 30 23:12:06.654750 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:06.654730    5896 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"]
Oct 30 23:12:06.657153 ip-10-0-138-202 kubenswrapper[5896]: W1030 23:12:06.657126    5896 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6f7dc650_8b4a_4af1_ae9c_1936ab8b4635.slice/crio-c50184594d48d3b3d998420375f8862d4c8a7858e458e9e3cfdd8f0ad0bd2f7d WatchSource:0}: Error finding container c50184594d48d3b3d998420375f8862d4c8a7858e458e9e3cfdd8f0ad0bd2f7d: Status 404 returned error can't find the container with id c50184594d48d3b3d998420375f8862d4c8a7858e458e9e3cfdd8f0ad0bd2f7d
Oct 30 23:12:07.046582 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:07.046553    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-pvxcb_fc8d2282-fc99-44d1-9d74-00eb93c5a7ec/monitoring-plugin/0.log"
Oct 30 23:12:07.491304 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:07.491271    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g" event={"ID":"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635","Type":"ContainerStarted","Data":"f1055ac27a9ece885eaf729edee4f1213ca92286d8bdb9c432de925432b981d3"}
Oct 30 23:12:07.491477 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:07.491349    5896 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g" event={"ID":"6f7dc650-8b4a-4af1-ae9c-1936ab8b4635","Type":"ContainerStarted","Data":"c50184594d48d3b3d998420375f8862d4c8a7858e458e9e3cfdd8f0ad0bd2f7d"}
Oct 30 23:12:07.491477 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:07.491372    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:07.610755 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:07.610707    5896 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g" podStartSLOduration=2.610693058 podStartE2EDuration="2.610693058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 23:12:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:12:07.609523656 +0000 UTC m=+3780.020989622" watchObservedRunningTime="2026-10-30 23:12:07.610693058 +0000 UTC m=+3780.022159025"
Oct 30 23:12:08.734031 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:08.734006    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hq4cs_7759bd31-01d0-41bf-a1ee-4a9ddb8221d0/node-exporter/0.log"
Oct 30 23:12:09.150624 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:09.150539    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hq4cs_7759bd31-01d0-41bf-a1ee-4a9ddb8221d0/kube-rbac-proxy/0.log"
Oct 30 23:12:09.567284 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:09.567257    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hq4cs_7759bd31-01d0-41bf-a1ee-4a9ddb8221d0/init-textfile/0.log"
Oct 30 23:12:11.248379 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:11.248342    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-sm67h_c8bebbb6-d965-4af2-91ff-09497cdbe81e/kube-rbac-proxy-main/0.log"
Oct 30 23:12:11.665131 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:11.665033    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-sm67h_c8bebbb6-d965-4af2-91ff-09497cdbe81e/kube-rbac-proxy-self/0.log"
Oct 30 23:12:12.082347 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:12.082325    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-sm67h_c8bebbb6-d965-4af2-91ff-09497cdbe81e/openshift-state-metrics/0.log"
Oct 30 23:12:12.519133 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:12.519110    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_350f2760-583f-4109-9d15-b4a82ac2d43a/prometheus/0.log"
Oct 30 23:12:12.933231 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:12.933151    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_350f2760-583f-4109-9d15-b4a82ac2d43a/config-reloader/0.log"
Oct 30 23:12:13.351984 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:13.351954    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_350f2760-583f-4109-9d15-b4a82ac2d43a/thanos-sidecar/0.log"
Oct 30 23:12:13.506094 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:13.506072    5896 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dk65g/perf-node-gather-daemonset-87h5g"
Oct 30 23:12:13.769407 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:13.769386    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_350f2760-583f-4109-9d15-b4a82ac2d43a/kube-rbac-proxy-web/0.log"
Oct 30 23:12:14.187434 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:14.187372    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_350f2760-583f-4109-9d15-b4a82ac2d43a/kube-rbac-proxy/0.log"
Oct 30 23:12:14.605039 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:14.605010    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_350f2760-583f-4109-9d15-b4a82ac2d43a/kube-rbac-proxy-thanos/0.log"
Oct 30 23:12:15.028034 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:15.028005    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_350f2760-583f-4109-9d15-b4a82ac2d43a/init-config-reloader/0.log"
Oct 30 23:12:15.451300 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:15.451202    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-4v9pw_7eac1f43-42ac-4e65-be08-ca44155cee2d/prometheus-operator/0.log"
Oct 30 23:12:15.869366 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:15.869342    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-4v9pw_7eac1f43-42ac-4e65-be08-ca44155cee2d/kube-rbac-proxy/0.log"
Oct 30 23:12:16.716339 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:16.716304    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cffccb95d-6vlcb_7cec808e-f1bb-4d43-98da-5779799ae953/telemeter-client/0.log"
Oct 30 23:12:17.133987 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:17.133924    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cffccb95d-6vlcb_7cec808e-f1bb-4d43-98da-5779799ae953/reload/0.log"
Oct 30 23:12:17.550498 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:17.550474    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cffccb95d-6vlcb_7cec808e-f1bb-4d43-98da-5779799ae953/kube-rbac-proxy/0.log"
Oct 30 23:12:17.977009 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:17.976979    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d45fc54c5-cz6fw_414ea994-62c2-4250-9c5a-787cddd4b61e/thanos-query/0.log"
Oct 30 23:12:18.394411 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:18.394345    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d45fc54c5-cz6fw_414ea994-62c2-4250-9c5a-787cddd4b61e/kube-rbac-proxy-web/0.log"
Oct 30 23:12:18.811633 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:18.811606    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d45fc54c5-cz6fw_414ea994-62c2-4250-9c5a-787cddd4b61e/kube-rbac-proxy/0.log"
Oct 30 23:12:19.230968 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:19.230943    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d45fc54c5-cz6fw_414ea994-62c2-4250-9c5a-787cddd4b61e/prom-label-proxy/0.log"
Oct 30 23:12:19.647459 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:19.647380    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d45fc54c5-cz6fw_414ea994-62c2-4250-9c5a-787cddd4b61e/kube-rbac-proxy-rules/0.log"
Oct 30 23:12:20.065594 ip-10-0-138-202 kubenswrapper[5896]: I1030 23:12:20.065567    5896 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d45fc54c5-cz6fw_414ea994-62c2-4250-9c5a-787cddd4b61e/kube-rbac-proxy-metrics/0.log"